REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  NO.  0704-0188 


The  public  reporting  burden  for  this  coilection  of  information  is  estimated  to  average  1  hour  per  response,  inciuding  the  time  for  reviewing  instructions, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  compieting  and  reviewing  the  coiiection  of  information.  Send  comments 
regarding  this  burden  estimate  or  any  other  aspect  of  this  coilection  of  information,  including  suggesstions  for  reducing  this  burden,  to  Washington 
Headquarters  Services,  Directorate  for  information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Ariington  VA,  22202-4302. 
Respondents  shouid  be  aware  that  notwithstanding  any  other  provision  of  iaw,  no  person  shaii  be  subject  to  any  oenaity  for  failing  to  comply  with  a  coiiection 
of  information  if  it  does  not  dispiay  a  currentiy  vaiid  OMB  controi  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


2.  REPORT  TYPE 
Final  Report 


1.  REPORT  DATE  (DD-MM-YYYY) 
23-04-2016 


4.  TITLE  AND  SUBTITLE  5a.  CONTRACT  NUMBER 

Final  Report:  Large-Seale  Partial-Duplicate  Image  Retrieval  and  W91  lNF-12-1-0057 
Its  Applications  5b.  GRANT  NUMBER 


3.  DATES  COVERED  (From  -  To) 
23-Jan-2012  -  22-Jan-2016 


6.  AUTHORS 
Yijuan  Lu,  Qi  Tian 


5c.  PROGRAM  ELEMENT  NUMBER 
206022 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAMES  AND  ADDRESSES 

Texas  State  University 

60 1  University  Drive,  JCK  420 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


San  Marcos,  TX  78666  -4864 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS 
(ES) 

U.S.  Army  Research  Office 
P.O.Box  1221 1 

Research  Triangle  Park,  NC  27709-2211 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 
ARO 


II.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

60497-CS-REP.126 


12.  DISTRIBUTION  AVAILIBILITY  STATEMENT 
Approved  for  Public  Release;  Distribution  Unlimited 


13.  SUPPLEMENTARY  NOTES 

The  views,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and  should  not  contmed  as  an  official  Department 
of  the  Army  position,  policy  or  decision,  unless  so  designated  by  other  documentation. 


14.  ABSTRACT 

The  explosive  growth  of  Internet  Media  (partial-duplicate/similar  images,  3D  objects,  3D  models,  etc.)  sheds  bright 
light  on  many  promising  applications  in  forensics,  surveillance,  3D  animation,  mobile  visual  search,  and  3D 
model/object  search.  Compared  with  the  general  images,  partial-duplicate  images  have  some  intrinsic  properties 
such  as  high  repeatability  of  local  features,  consistent  local  patch  appearance,  and  stable  spatial  configuration. 
Compared  with  the  general  2D  objects,  3D  models/objects  consist  of  3D  data  information  (typically  a  list  of 


15.  SUBJECT  TERMS 

Multimedia  Information  Retrieval,  Computer  Vision 


_ j 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

15.  NUMBER 

a.  REPORT 

b.  ABSTRACT 

e.  THIS  PAGE 

ABSTRACT 

OF  PAGES 

UU 

UU 

UU 

UU 

Yijuan  Lu 


19b.  TELEPHONE  NUMBER 
512-245-6580 


Standard  Form  298  (Rev  8/98) 
Prescribed  by  ANSI  Std.  Z39. 18 


Report  Title 

Final  Report:  Large-Scale  Partial-Duplicate  Image  Retrieval  and  Its  Applications 

ABSTRACT 

The  explosive  growth  of  Internet  Media  (partial-duplieate/similar  images,  3D  objeets,  3D  models,  ete.)  sheds  bright  light  on  many  promising 
applieations  in  forensies,  surveillanee,  3D  animation,  mobile  visual  seareh,  and  3D  model/objeet  seareh.  Compared  with  the  general  images, 
partial-duplieate  images  have  some  intrinsie  properties  sueh  as  high  repeatability  of  loeal  features,  eonsistent  loeal  pateh  appearanee,  and 
stable  spatial  eonfiguration.  Compared  with  the  general  2D  objeets,  3D  models/objeets  eonsist  of  3D  data  information  (typieally  a  list  of 
vertiees  and  faees)  to  represent  3D  objeets.  However,  these  unique  properties  of  partial-duplieate  images  and  3D  models  have  not  been  well 
exploited  to  design  effeetive  and  effieient  seareh  algorithms.  Beeause  of  this,  existing  works  for  large-seale  partial-duplieate  image  retrieval 
and  3D  model  retrieval  suffer  from  two  major  problems:  low  aeeuraey  and  low  effieieney.  These  problems  make  them  fall  far  below  many 
applieations’  requirement.  This  projeet  has  investigated  many  key  problems  in  large-seale  partial-duplieate/similar  image  and  3D  model 
retrieval:  feature  deseriptor  problem,  image  representation  problem,  index  strategy  problem,  feature  quantization  problem,  image  seareh 
results  quality  assessment  problem,  image  seareh  reranking  problem,  sketeh-based  3D  model  retrieval  problem,  and  related  seareh  problems 
and  has  proposed  a  series  of  effeetive  and  effieient  approaehes  to  solve  them. 


Enter  List  of  papers  submitted  or  published  that  acknowledge  ARO  support  from  the  start  of 
the  project  to  the  date  of  this  printing.  List  the  papers,  including  journal  references,  in  the 
following  categories: 

(a)  Papers  published  in  peer-reviewed  journals  (N/A  for  none) 


Received  Paper 


04/22/2016  96.00 

04/22/2016  09.00 

04/22/2016  08.00 

04/22/2016  06.00 

04/22/2016  05.00 

04/22/2016  04.00 

04/22/2016  03.00 

04/22/2016  02.00 

04/22/2016  01.00 

04/22/2016  00.00 

04/22/2016  99.00 

04/22/2016  98.00 

04/22/2016  97.00 


Liang  Zheng,  Shengjin  Wang,  Peizhen  Guo,  Hanyue  Liang,  Qi  Tian.  Tensor  index  for  large  scale  image 
retrieval. 

Multimedia  Systems,  (10  2014):  0.  doi:  10.1007/s00530-014-0415-8 

Chunjie  Zhang,  Xiaobin  Zhu,  Liang  Li,  Yifan  Zhang,  Jing  Liu,  Qingming  Huang,  Qi  Tian.  Joint  image 
representation  and  classification  in  random  semantic  spaces. 

Neurocomputing,  (05  2015):  0.  doi:  10.1016/j.neucom.2014.12.083 

Shanmin  Pang,  Jianru  Xue,  Zhanning  Gao,  Qi  Tian.  Image  re-ranking  with  an  alternating  optimization. 
Neurocomputing,  (10  2015):  0.  doi:  10.1016/j.neucom.2015.03.040 

Luming  Zhang,  Meng  Wang,  Liqiang  Nie,  Liang  Hong,  Yong  Rui,  Qi  Tian.  Retargeting  Semantically-Rich 
Photos, 

IEEE  Transactions  on  Multimedia,  (09  2015):  0.  doi:  10.1 109/TMM. 2015. 2451954 

Lingxi  Xie,  Qi  Tian,  Wengang  Zhou,  Bo  Zhang.  Heterogeneous  Graph  Propagation  for  Large-Scale  Web 
Image  Search, 

IEEE  Transactions  on  Image  Processing,  (1 1  2015):  0.  doi:  10.1 109/TIP. 2015. 2432673 

Lei  Zhang,  Yongdong  Zhang,  Richang  Hong,  Qi  Tian.  Full-Space  Local  Topology  Extraction  for  Cross- 
Modal  Retrieval, 

IEEE  Transactions  on  Image  Processing,  (07  2015):  0.  doi:  10. 1109/TIP. 2015. 2419074 

Shiliang  Zhang,  Xiaoyu  Wang,  Yuanqing  Lin,  Qi  Tian.  Cross  Indexing  With  Grouplets, 

IEEE  Transactions  on  Multimedia,  (1 1  2015):  0.  doi:  10.1 109/TMM. 2015. 2478055 

Lingxi  Xie,  Qi  Tian,  Bo  Zhang.  Simple  Techniques  Make  Sense:  Feature  Pooling  and  Normalization  for 
Image  Classification, 

IEEE  Transactions  on  Circuits  and  Systems  for  Video  Technology,  (06  2015):  0.  doi:  10.1109/TCSVT. 
2015.2461978 

Xinmei  Tian,  Yijuan  Lu,  Nate  Stender,  Linjun  Yang,  Dacheng  Tao.  Exploration  of  Image  Search  Results 
Quality  Assessment, 

IEEE  Transactions  on  Bigdata,  (09  2015):  0.  doi:  10.1109/TBDATA.201 5.2497710 

Wengang  Zhou,  Ming  Yang,  Houqiang  Li,  Xiaoyu  Wang,  Yuanqing  Lin,  Qi  Tian.  Towards  Codebook-Free: 
Scalable  Cascaded  Hashing  for  Mobile  Image  Search, 

IEEE  Transactions  on  Multimedia,  (04  2014):  0.  doi:  10.1 109/TMM.2014.2301979 

Yongdong  Zhang,  Lei  Zhang,  Qi  Tian.  A  Prior-Free  Weighting  Scheme  for  Binary  Code  Ranking, 

IEEE  Transactions  on  Multimedia,  (06  2014):  0.  doi:  10.1 109/TMM.2014.2306392 

Lei  Zhang,  Yongdong  Zhang,  Xiaoguang  Gu,  Jinhui  Tang,  Qi  Tian.  Scalable  Similarity  Search  With 
Topology  Preserving  Hashing, 

IEEE  Transactions  on  Image  Processing,  (07  2014):  0.  doi:  10.1 109/TIP. 2014. 2326010 

Lingxi  Xie,  Qi  Tian,  Wengang  Zhou,  Bo  Zhang.  Fast  and  accurate  near-duplicate  image  search  with 
affinity  propagation  on  the  ImageWeb, 

Computer  Vision  and  Image  Understanding,  (07  2014):  0.  doi:  10.1016/j.cviu.2013.12.011 


08/30/2013  7.00  Houqiang  Li,  Yijuan  Lu,  Meng  Wang,  Qi  Tian,  Wengang  Zhou.  Visual  word  expansion  and  BSIFT 
verification  for  large-scale  image  search. 

Multimedia  Systems,  (07  2013):  0.  doi:  10.1007/s00530-01 3-0330-4 

08/30/2013  11  00  Xinmei  Tian,  Yijuan  Lu,  Linjun  Yang.  Query  Difficulty  Prediction  for  Web  Image  Search, 

IEEE  Transactions  on  Multimedia,  (08  2012):  0.  doi:  10.1 109/TMM.201 1 .2177647 

08/30/2013  10.00  Houqiang  Li,  Yijuan  Lu,  Wengang  Zhou,  Qi  Tian.  Principal  Visual  Word  Discovery  for  Automatic  License 
Plate  Detection, 

IEEE  Transactions  on  Image  Processing,  (09  2012):  0.  doi:  10.1 109/TIP. 2012. 2199506 

08/30/2013  9.00  Xinmei  Tian,  Yijuan  Lu.  Discriminative  codebook  learning  for  Web  image  search. 

Signal  Processing,  (08  2013):  0.  doi:  10.1016/j.sigpro.2012.04.018 

08/30/2013  8.00  Wengang  Zhou,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  SIFT  match  verification  by  geometric  coding  for  large- 
scale  partial-duplicate  web  image  search, 

ACM  Transactions  on  Multimedia  Computing,  Communications  and  Applications  ,  (02  2013):  0.  doi: 
10.1145/2422956.2422960 

08/31/2013  13.00  Bingbing  Ni,  Mengdi  Xu,  Bin  Cheng,  Meng  Wang,  Shuicheng  Yan,  Qi  Tian.  Learning  to  Photograph:  A 
Compositional  Perspective, 

IEEE  Transactions  on  Multimedia,  (08  2013):  0.  doi:  10.1 109/TMM.2013.2241042 

08/31/2013  12.00  Shiliang  Zhang,  Qi  Tian,  Ke  Lu,  Qingming  Huang,  Wen  Gao.  Edge-SIFT:  Discriminative  Binary  Descriptor 
for  Scalable  Partial-Duplicate  Mobile  Search, 

IEEE  Transactions  on  Image  Processing,  (07  2013):  0.  doi:  10.1 109/TIP. 2013. 2251650 

08/31/2013  14.00  Bingbing  Ni,  Shuicheng  Yan,  Meng  Wang,  A.  A.  Kassim,  Qi  Tian.  High-Qrder  Local  Spatial  Context 
Modeling  by  Spatialized  Random  Forest, 

IEEE  Transactions  on  Image  Processing,  (02  2013):  0.  doi:  10.1 109/TIP. 2012. 2222895 

08/31/2013  15.00  Binxing  Jiao,  Linjun  Yang,  Jizheng  Xu,  Qi  Tian,  Feng  Wu.  Visually  Summarizing  Web  Pages  Through 
Internal  and  External  Images, 

IEEE  Transactions  on  Multimedia,  (12  2012):  0.  doi:  10.1 109/TMM.2012.2198457 

08/31/2013  16.00  Shuhui  Wang,  Qingming  Huang,  Shuqiang  Jiang,  Qi  Tian.  S3MKL:  Scalable  Semi-Supervised  Multiple 
Kernel  Learning  for  Real-World  Image  Applications, 

IEEE  Transactions  on  Multimedia,  (08  2012):  0.  doi:  10.1 109/TMM.2012.2193120 

08/31/2015  69.00  Bo  Li,  Yijuan  Lu,  Chunyuan  Li,  Afzal  Godil,  Tobias  Schreck,  Masaki  Aono,  Martin  Burtscher,  Qiang  Chen, 
Nihad  Karim  Chowdhury,  Bin  Fang,  Hongbo  Fu,  Takahiko  Furuya,  Haisheng  Li,  Jianzhuang  Liu,  Henry 
Johan,  Ryuichi  Kosaka,  Hitoshi  Koyanagi,  Ryutarou  Qhbuchi,  Atsushi  Tatsuma,  Yajuan  Wan,  Chaoli 
Zhang,  Changqing  Zou.  A  comparison  of  3D  shape  retrieval  methods  based  on  a  large-scale  benchmark 
supporting  multimodal  queries. 

Computer  Vision  and  Image  Understanding,  (02  2015):  0.  doi:  10.1016/j.cviu.2014.10.006 

08/31/2015  78.00  Shiliang  Zhang,  Ming  Yang,  Xiaoyu  Wang,  Yuanqing  Lin,  Qi  Tian.  Semantic-aware  Co-indexing  for  Image 
Retrieval, 

IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence,  (03  2015):  0.  doi:  10.1 109/TPAMI. 
2015.2417573 

08/31/2015  77.00  Wengang  Zhou,  Ming  Yang,  Xiaoyu  Wang,  Houqiang  Li,  Yuanqing  Lin,  Qi  Tian.  Scalable  Feature 
Matching  by  Dual  Cascaded  Scalar  Quantization  for  Image  Retrieval, 

IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence,  (05  2015):  0.  doi:  10.1 109/TPAMI. 
2015.2430329 

08/31/2015  76.00  Zhen  Liu,  Houqiang  Li,  Wengang  Zhou,  Ting  Rui,  Qi  Tian.  Uniforming  Residual  Vector  Distribution  for 
Distinctive  Image  Representation, 

IEEE  Transactions  on  Circuits  and  Systems  for  Video  Technology,  (03  2015):  0.  doi:  10.1109/TCSVT. 
2015.2409693 

08/31/2015  74.00  Jingdong  Wang,  Bo  Zhang,  Qi  Tian,  Lingxi  Xie.  Fine-Grained  Image  Search, 

IEEE  Transactions  on  Multimedia,  (05  2015):  0.  doi:  10.1 109/TMM. 2015. 2408566 


08/31/2015  73.00  Liang  Zheng,  Shengjin  Wang,  Ziqiong  Liu,  Qi  Tian.  Fast  Image  Retrieval:  Query  Pruning  and  Early 
Termination, 

IEEE  Transactions  on  Multimedia,  (05  2015):  0.  doi:  10.1 109/TMM. 2015. 2408563 

08/31/2015  75.00  Zhen  Liu,  Houqiang  Li,  Wengang  Zhou,  Richang  Hong,  Qi  Tian.  Uniting  Keypoints:  Local  Visual 
Information  Fusion  for  Large-Scale  Image  Search, 

IEEE  Transactions  on  Multimedia,  (04  2015):  0.  doi:  10.1 109/TMM. 2015. 2399851 

08/31/2015  72.00  Junjie  Cai,  Zheng-Jun  Zha,  Meng  Wang,  Shiliang  Zhang,  Qi  Tian.  An  Attribute-Assisted  Reranking  Model 
for  Web  Image  Search, 

IEEE  Transactions  on  Image  Processing,  (01  2015):  0.  doi:  10.1 109/TIP. 2014. 2372616 

08/31/2015  71.00  Houqiang  Li,  Richang  Hong,  Yijuan  Lu,  Qi  Tian,  Wengang  Zhou.  BSIFT:  Toward  Data-Independent 
Codebook  for  Large  Scale  Image  Search, 

IEEE  Transactions  on  Image  Processing,  (03  2015):  0.  doi:  10.1 109/TIP. 2015. 2389624 

08/31/2015  70.00  Xinmei  Tian,  Linjun  Yang,  Yijuan  Lu,  Qi  Tian,  Dacheng  Tao.  Image  Search  Reranking  With  Hierarchical 
Topic  Awareness, 

IEEE  Transactions  on  Cybernetics,  (01  2015):  0.  doi:  10.1109/TCYB. 2014. 2366740 

09/01/2014  38.00  Yijuan  Lu,  Bo  Li,  Afzal  Godil,  Tobias  Schreck,  Benjamin  Bustos,  Alfredo  Ferreira,  Takahiko  Furuya, 
Manuel  J.  Fonseca,  Henry  Johan,  Takahiro  Matsuda,  Ryutarou  Qhbuchi,  Pedro  B.  Pascoal,  Jose  M. 
Saavedra.  A  comparison  of  methods  for  sketch-based  3D  shape  retrieval. 

Computer  Vision  and  Image  Understanding,  (02  2014):  0.  doi:  10.1016/j.cviu.2013.1 1 .008 

09/01/2014  39.00  Benjamin  Bustos,  Tobias  Schreck,  Bo  Li,  Yijuan  Lu,  Ivan  Sipiran,  Henry  Johan,  Rafael  Meruane.  A 
benchmark  of  simulated  range  images  for  partial  shape  retrieval. 

The  Visual  Computer,  (05  2014):  0.  doi:  10.1007/s00371-014-0937-2 

09/01/2014  40.00  Shiliang  Zhang,  Qi  Tian,  Qingming  Huang,  Wen  Gao,  Yong  Rui.  USB:  Ultrashort  Binary  Descriptor  for 
Fast  Visual  Matching  and  Retrieval, 

IEEE  Transactions  on  Image  Processing,  (08  2014):  0.  doi:  10.1 109/TIP. 2014. 2330794 

09/01/2014  41.00  Shengjin  Wang,  Qi  Tian,  Liang  Zheng.  Coupled  Binary  Embedding  for  Large-Scale  Image  Retrieval, 

IEEE  Transactions  on  Image  Processing,  (08  2014):  0.  doi:  10.1 109/TIP. 2014. 2330763 

09/01/2014  42.00  Liang  Zheng,  Shengjin  Wang,  Qi  Tian.  Lp-norm  IDF  for  Scalable  Image  Retrieval, 

IEEE  Transactions  on  Image  Processing,  (08  2014):  0.  doi:  10.1 109/TIP. 2014. 2329182 

09/01/2014  43.00  Qi  Tian,  Qingming  Huang,  Wen  Gao,  Shiliang  Zhang,  Yong  Rui.  Cascade  Category-Aware  Visual  Search, 
IEEE  Transactions  on  Image  Processing,  (06  2014):  0.  doi:  10.1 109/TIP. 2014. 2317986 

09/01/2014  44.00  Qi  Tian,  Meng  Wang,  Bo  Zhang,  Lingxi  Xie.  Spatial  Pooling  of  Heterogeneous  Features  for  Image 
Classification, 

IEEE  Transactions  on  Image  Processing,  (05  2014):  0.  doi:  10.1 109/TIP. 2014. 23101 17 

09/01/2014  45.00  Zhen  Liu,  Houqiang  Li,  Liyan  Zhang,  Wengang  Zhou,  Qi  Tian.  Cross-Indexing  of  Binary  SIFT  Codes  for 
Large-Scale  Image  Search, 

IEEE  Transactions  on  Image  Processing,  (05  2014):  0.  doi:  10.1 109/TIP. 2014. 2312283 

09/01/2014  46.00  Zhen  Liu,  Houqiang  Li,  Wengang  Zhou,  Ruizhen  Zhao,  Qi  Tian.  Contextual  Hashing  for  Large-Scale 
Image  Search, 

IEEE  Transactions  on  Image  Processing,  (04  2014):  0.  doi:  10.1 109/TIP. 2014. 2305072 

09/01/2014  47.00  Shiliang  Zhang,  Qi  Tian,  Qingming  Huang,  Yong  Rui.  Embedding  Multi-Qrder  Spatial  Clues  for  Scalable 
Visual  Matching  and  Retrieval, 

IEEE  Journal  on  Emerging  and  Selected  Topics  in  Circuits  and  Systems,  (03  2014):  0.  doi:  10.1109 
/JETCAS.2014.2298272 

09/01/2014  48.00  Shiliang  Zhang,  Qi  Tian,  Gang  Hua,  Qingming  Huang,  Wen  Gao.  QbjectPatchNet:  Towards  scalable  and 
semantic  image  annotation  and  retrieval. 

Computer  Vision  and  Image  Understanding,  (01  2014):  0.  doi:  10.1016/j.cviu.2013.03.008 


12/10/2015  49.00  Wengang  Zhou,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  Encoding  Spatial  Context  for  Large  Scale  Partial- 
Duplicate, 

Journal  of  Computer  Science  and  Technology,  (09  2014):  837.  doi: 

TOTAL:  45 

Number  of  Papers  published  iu  peer-reviewed  jouruals: 

(b)  Papers  published  in  non-peer-reviewed  journals  (N/A  for  none) 

Received  Paper 

TOTAL: 

Number  of  Papers  published  iu  uou  peer-reviewed  jouruals: 


(c)  Presentations 


Number  of  Preseutatious:  0.00 


Non  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


Received  Paper 


TOTAL: 


Number  of  Non  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


Received  Paper 


04/22/2016  07.00  Shanmin  Pang,  Jianru  Xue,  Zhanning  Gao,  Qi  Tian.  Image  Re-ranking  with  an  Alternating  Optimization, 
the  ACM  International  Conference.  02-NOV-14,  Orlando,  Florida,  USA.  :  , 

04/22/2016  25.00  Shiliang  Zhang,  Ming  Yang,  Xiaoyu  Wang,  Yuanqing  Lin,  Oi  Tian.  Semantic-Aware  Co-indexing  for  Image 
Retrieval, 

2013  IEEE  International  Conference  on  Computer  Vision  (ICCV).  30-NOV-13,  Sydney,  Australia.  :  , 

04/22/2016  24.00  Guoli  Song,  Shuhui  Wang,  Oi  Tian.  Fusing  feature  and  similarity  for  multimodal  search, 

2015  IEEE  China  Summit  and  International  Conference  on  Signal  and  Information  Processing  (ChinaSIP). 
11-JUL-15,  Chengdu,  China.  :  , 

04/22/2016  23.00  Liang  Zheng,  Shengjin  Wang,  Oi  Tian.  Coloring  image  search  with  coupled  multi-index, 

2015  IEEE  China  Summit  and  International  Conference  on  Signal  and  Information  Processing  (ChinaSIP). 
11-JUL-15,  Chengdu,  China.  :  , 

04/22/2016  22.00  Lingxi  Xie,  Richang  Hong,  Bo  Zhang,  Oi  Tian.  Image  Classification  and  Retrieval  are  ONE, 
the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

04/22/2016  21.00  Hantao  Yao,  Shiliang  Zhang,  Fei  Xie,  Yongdong  Zhang,  Dongming  Zhang,  Yu  Su,  Oi  Tian.  Orientational 
Spatial  Part  Modeling  for  Fine-Grained  Visual  Categorization, 

2015  IEEE  International  Conference  on  Mobile  Services  (MS).  26-JUN-15,  New  York  City,  NY,  USA.  :  , 

04/22/2016  20.00  Xu  Xie,  Wengang  Zhou,  Houqiang  Li,  Tian  Oi.  Rank-aware  graph  fusion  with  contextual  dissimilarity 
measurement  for  image  retrieval, 

2015  IEEE  International  Conference  on  Image  Processing  (ICIP).  26-SEP-15,  Ouebec  City,  OC,  Canada. 


04/22/2016  19.00  Jingdong  Wang,  Weiyao  Lin,  Bo  Zhang,  Oi  Tian,  Lingxi  Xie.  RIDE:  Reversal  Invariant  Descriptor 
Enhancement, 

2015  IEEE  International  Conference  on  Computer  Vision  (ICCV).  06-DEC-15,  Santiago,  Chile.  :  , 

04/22/2016  18.00  Oingjun  Luo,  Shiliang  Zhang,  Tiejun  Huang,  Wen  Gao,  Oi  Tian.  Superimage:  Packing  Semantic-Relevant 
Images  for  Indexing  and  Retrieval, 

International  Conference.  31-MAR-14,  Glasgow,  United  Kingdom.  :  , 

04/22/2016  17.00  Ziqiong  Liu,  Shengjin  Wang,  Liang  Zheng,  Oi  Tian.  Visual  reranking  with  improved  image  graph, 

ICASSP  2014  -  2014  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing 
(ICASSP).  03-MAY-14,  Florence,  Italy.  :  , 

04/22/2016  16.00  Guangxin  Ren,  Junjie  Cai,  Shipeng  Li,  Nenghai  Yu,  Oi  Tian.  Salable  Image  Search  with  Reliable  Binary 
Code, 

the  ACM  International  Conference.  02-NOV-14,  Orlando,  Florida,  USA.  :  , 

04/22/2016  15.00  Zhenxing  Niu,  Shiliang  Zhang,  Xinbo  Gao,  Oi  Tian.  Personalized  Visual  Vocabulary  Adaption  for  Social 
Image  Retrieval, 

the  ACM  International  Conference.  02-NOV-14,  Orlando,  Florida,  USA.  :  , 

04/22/2016  14.00  Xiaopeng  Zhang,  Hongkai  Xiong,  Wengang  Zhou,  Oi  Tian.  Fused  one-vs-all  mid-level  features  for  fine¬ 
grained  visual  categorization, 

the  ACM  International  Conference.  02-NOV-14,  Orlando,  Florida,  USA.  :  , 


04/22/2016  13.00  Shaowei  Liu,  Peng  Cui,  Wenwu  Zhu,  Shiqiang  Yang,  Qi  Tian.  Social  Embedding  Image  Distance 
Learning, 

the  ACM  International  Conference.  02-NOV-14,  Orlando,  Florida,  USA.  :  , 

04/22/2016  12.00  Qingjun  Luo,  Shiliang  Zhang,  Tiejun  Huang,  Wen  Gao,  Qi  Tian.  Scalable  mobile  search  with  binary 
phrase, 

the  Fifth  International  Conference.  16-AUG-13,  Huangshan,  China.  :  , 

04/22/2016  11.00  Shanmin  Pang,  Jianru  Xue,  Nanning  Zheng,  Qi  Tian.  Locality  preserving  verification  for  image  search, 
the  21st  ACM  international  conference.  20-OCT-13,  Barcelona,  Spain.  :  , 

04/22/2016  10.00  Lei  Zhang,  Yongdong  Zhang,  Jinhui  Tang,  Xiaoguang  Gu,  Jintao  Li,  Qi  Tian.  Topology  preserving  hashing 
for  similarity  search, 

the  21st  ACM  international  conference.  20-QCT-13,  Barcelona,  Spain.  :  , 

08/30/2012  1.00  Wengang  Zhou,  Yijuan  Lu,  Houqiang  Li,  Qi  Tian.  Scalar  Quantization  for  Large  Scale  Image  Search, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/30/2012  6.00  Wengang  Zhou,  Jinghui  Tang,  Qi  Tian,  Xia  Li.  Query  Expansion  Enhancement  by  Fast  Binary  Matching, 
ACM  Multimedia.  29-QCT-12,  .  :  , 

08/30/2012  5.00  Wengang  Zhou,  Meng  Wang,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  Binary  SIFT:  Towards  Efficient  Feature 
Matching  Verification  for  Image  Search, 

International  Conference  on  Internet  Multimedia  Computing  and  Service.  09-SEP-12,  .  :  , 

08/31/2013  17.00  Shaomin  Fang,  Yijuan  Lu,  Xinmei  Tian.  Learning  Image  Saliency  from  Human  Touch  Behaviors, 

IEEE  International  Conference  on  Multimedia  and  Expo  .  15-JUL-13,  .  :  , 

08/31/2013  37.00  Jianqiu  Ji  ^  Jianmin  Li Shuicheng  Yan,  Bo  Zhang  ^  Qi  Tian.  Super-Bit  Locality-Sensitive  Hashing, 

The  Neural  Information  Processing  Systems.  03-DEC-12,  .  :  , 

08/31/2013  36.00  Shiliang  Zhang,  Qi  Tian,  Qingming  Huang,  Wen  Gao,  Yong  Rui.  Multi-order  visual  phrase  for  scale  image 
search. 

International  Conference  on  Internet  Multimedia  Computing  and  Service.  17-AUG-13,  .  :  , 

08/31/2013  35.00  Junjie  Cai,  Zheng-Jun  Zha,  Wengang  Zhou,  Qi  Tian.  Attribute-assisted  Reranking  for  Web  Image 
Retrieval, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/31/2013  34.00  Jie  Xiao,  Wengang  Zhou,  Xia  Li,  Meng  Wang,  Qi  Tian.  Image  Tag  Re-ranking  by  Coupled  Probability 
Transition, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/31/2013  33.00  Yahong  Han,  Fei  Wu,  Xinyan  Lu,  Qi  Tian,  Yueting  Zhuang,  Jiebo  Luo.  Correlated  Attribute  Transfer  with 
Multi-task  Graph-Guided  Fusion, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/31/2013  32.00  Zhen  Liu,  Houqiang  Li,  Wengang  Zhou,  Qi  Tian.  Embedding  Spatial  Context  into  Inverted  File  for  Large- 
Scale  Image  Search, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/31/2013  31 .00  Qi  Tian,  Bo  Zhang,  Lingxi  Xie.  Spatial  Pooling  of  Heterogeneous  Features  for  Image  Applications, 

ACM  Multimedia.  29-QCT-12,  .  :  , 

08/31/2013  29.00  Dongming  Zhang,  Jintao  Li,  Qi  Tian,  Dongye  Zhuang.  Novel  Binary  Feature  from  Intensity  Difference 
Quantization  between  Random  Sample  of  Points, 

19th  International  Conference  on  Multimedia  Modeling.  07-JAN-13,  .  :  , 


08/31/2013  28.00  Shaowei  Liu,  Peng  Cui,  Huanbo  Luan,  Wenwu  Zhu,  Shiqiang  Yang,  Qi  Tian.  Social  visual  image  ranking 
for  Web  image  search. 

International  Conference  on  Multimedia  Modeling.  07-JAN-13,  .  :  , 

08/31/2013  27.00  Zheng-Jun  Zha,  Huanbo  Luan,  Shiliang  Zhang  ,  Qi  Tian,  Junjie  Cai.  Learning  Attribute-aware  Dictionary 
for  Image  Classification  and  Search, 

ACM  International  Conference  on  Multimedia  Retrieval.  16-APR-13,  .  :  , 

08/31/2013  26.00  Lei  Zhang,  Yongdong  Zhang,  Jinhui  Tang,  Ke  Lu,  Qi  Tian.  Binary  Code  Ranking  with  Weighted  Hamming 
Distance, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition  .  23-JUN-13,  .  :  , 

08/31/2013  25.00  Liang  Zheng,  Shengjin  Wang,  Ziqiong  Liu,  Qi  Tian.  Lp-norm  IDF  for  Large  Scale  Image  Search, 
International  Conference  on  Computer  Vision  and  Pattern  Recognition.  23-JUN-13,  .  :  , 

08/31/2013  24.00  Masaki  Aono,  Moumen  El-Melegy,  Aly  Farag,  Alfredo  Ferreiray,  Henry  Johan,  Bo  Li,  Yijuan  Lu,  Mostafa 
Abdelrahman,  Joao  Machadoy,  Pedro  B.  Pascoaly,  Atsushi  Tatsuma.  SHREC'13  Track:  Retrieval  of 
Qbjects  Captured  with  Low-Cost  Depth-Sensing  Cameras, 

The  6th  Eurographics  Workshop  on  3D  Qbject  Retrieval  .  11-MAY-13,  .  :  , 

08/31/2013  18.00  Yijuan  Lu,  Ribel  Fares,  Bo  Li.  Semantic  Sketch-based  3D  Model  Retrieval, 

IEEE  International  Conference  on  Multimedia  and  Expo.  15-JUL-13,  .  :  , 

08/31/2013  19.00  Yijuan  Lu,  Hao  Jiang.  Human  Movement  Summarization  and  Depiction  From  Videos, 

IEEE  International  Conference  on  Multimedia  and  Expo.  15-JUL-13,  .  :  , 

08/31/2013  20.00  Xinmei  Tian,  Linjun  Yang,  Yupeng  Yan,  Yijuan  Lu,  Houqiang  Li.  Semantic-Spatial  Matching  for  Image 
Classification, 

IEEE  International  Conference  on  Multimedia  and  Expo  .  15-JUL-13,  .  :  , 

08/31/2013  21.00  Bo  Li,  Yijuan  Lu,  Henry  Johan.  Sketch-based  3D  Model  Retrieval  By  Viewpoint  Entropy-based  Adaptive 
View  Clustering, 

The  6th  Eurographics  Workshop  on  3D  Qbject  Retrieval  .  11-MAY-13,  .  :  , 

08/31/2013  22.00  B.  Li  ,  Y.  Lu,  A.  Godil,  T.  Schreck,  M.  Aono,  H.  Johan,  J.  M.  Saavedra,  S.  Tashiro.  SHREC'13  Track: 
Large  Scale  Sketch-Based  3D  Shape  Retrieval, 

The  6th  Eurographics  Workshop  on  3D  Qbject  Retrieval  .  11-MAY-13,  .  :  , 

08/31/2015  79.00  Junjie  Cai,  Richang  Hong,  Meng  Wang,  Qi  Tian.  Exploring  feature  space  with  semantic  attributes, 

2015  IEEE  International  Conference  on  Multimedia  and  Expo  (ICME).  28-JUN-15,  Turin,  Italy.  :  , 

08/31/2015  95.00  Lingxi  Xie,  Richang  Hong,  Bo  Zhang,  Qi  Tian.  Image  Classification  and  Retrieval  are  QNE, 

International  Conference  of  Multimedia  Retriveal.  23-JUN-15,  .  :  , 

08/31/2015  90.00  Yang  Zhou,  Bingbing  Ni,  Richang  Hong,  Meng  Wang,  Qi  Tian.  Interaction  Part  Mining:  A  Mid-Level 
Approach  for  Fine-Grained  Action  Recognition, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  08-JUN-15,  .  :  , 

08/31/2015  88.00  Liang  Zheng,  Shengjin  Wang,  Lu  Tian,  Fei  He,  Ziqiong  Liu,  Qi  Tian.  Query-Adaptive  Late  Fusion  for 
Image  Search  and  Person  Re-identification, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  08-JUN-15,  .  :  , 

08/31/2015  89.00  Liang  Zheng,  Shengjin  Wang,  Lu  Tian,  Fei  He,  Ziqiong  Liu,  Qi  Tian.  Query-Adaptive  Late  Fusion  for 
Image  Search  and  Person  Re-identification, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  08-JUN-15,  .  :  , 

08/31/2015  87.00  Bo  Li,  Yijuan  Lu,  Azeem  Ghumman,  Bradley  Strylowski,  Mario  Gutierrez,  Safiyah  Sadiq,  Scott  Forster, 
Natacha  Feola,  Travis  Bugerin.  KinectSBR:  A  Kinect-Assisted  3D  Sketch-Based3D  Model  Retrieval 
System, 

the  5th  ACM  on  International  Conference  on  Multimedia  Retrieval.  22-JUN-15,  Shanghai,  China.  :  , 


08/31/2015  86.00  Bo  Li,  Yijuan  Lu,  Azeem  Ghumman,  Bradley  Strylowski,  Mario  Gutierrez,  Safiyah  Sadiq,  Scott  Forster, 
Natacha  Feola,  Travis  Bugerin.  3D  Sketch-Based  3D  Model  Retrieval, 
the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

08/31/2015  85.00  Bo  Li,  Henry  Johan,  Yuxiang  Ye,  Yijuan  Lu.  Efficient  view-based  3d  reflection  symmetry  detection, 
SIGGRAPH  Asia  2014  Creative  Shape  Modeling  and  Design.  02-DEC-14,  Shenzhen,  China.  :  , 

08/31/2015  84.00  Wengang  Zhou,  Shanmin  Pang,  Qi  Tian,  Zhanning  Gao,  Jianru  Xue.  Fast  Democratic  Aggregation  and 
Query  Fusion  for  Image  Search, 
the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

08/31/2015  83.00  Zhendong  Mao,  LingLing  Tong,  Hongtao  Xie,  Qi  Tian.  Hierarchical  Encoding  of  Binary  Descriptors  for 
Image  Matching, 

the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

08/31/2015  81.00  Dan  Zeng,  Yang  Zhou,  Shiliang  Zhang,  Qi  Tian.  Augmented  Feature  Fusion  for  Image  Retrieval  System, 
the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

08/31/2015  82.00  Sharath  Pankanti,  Qi  Tian,  Junjie  Cai,  Michele  Merler.  Heterogeneous  Semantic  Level  Features  Fusion 
for  Action  Recognition, 

the  5th  ACM.  22-JUN-15,  Shanghai,  China.  :  , 

09/01/2014  23.00  R.  Meruane,  B.  Bustos,  T.  Schreck,  H.  Johan,  B.  Li,  Y.  Lu,  1.  Sipiran.  SHREC'13  Track:  Large-Scale 
Partial  Shape  Retrieval  Track  Using  Simulated  Range  Images, 

The  6th  Eurographics  Workshop  on  3D  Qbject  Retrieval  .  11-MAY-13,  .  :  , 

09/01/2014  54.00  Bo  Li,  Y.  V.  Venkatesh,  Ashraf  Kassim,  Yijuan  Lu.  Improving  PMVS  Algorithm  For  3D 
SceneReconstruction  from  Sparse  Stereo  Pairs, 

Pacific-rim  Conference  on  Multimedia.  13-DEC-13,  .  :  , 

09/01/2014  55.00  Peng  Zhao  ,  Yijuan  Lu  ,  WenBin  Wang  ,  WeiWei  Zhu.  Automatic  Image  Annotation  using  Semantic 
Relevance, 

[2]  International  Conference  on  Internet  Multimedia  Computing  and  Service.  17-AUG-13,  .  :  , 

09/01/2014  56.00  B.  Li,  Y.  Lu,  C.  Li,  A.  Godil,  T.  Schreck,  M.  Aono,  M.  Burtscher,  H.  Fu,  T.  Furuya,  H.  Johan,  J.  Liu,  R. 

Qhbuchi,  A.  Tatsuma,  C.  Zou.  SHREC’14  Track:  Extended  Large  Scale  Sketch-Based  3DShape 
Retrieval, 

Eurographics  Workshop  on  3D  Qbject  Retrieval  2014.  06-APR-14,  .  :  , 

09/01/2014  57.00  B.  Li,  Y.  Lu,  C.  Li,  A.  Godil,  T.  Schreck,  M.  Aono,  Q.  Chen,  N.K.  Chowdhury,  B.  Fang,  T.  Furuya,  H. 
Johan,  R.  Kosaka,  H.  Koyanagi,  R.  Qhbuchi,  A.  Tatsuma.  SHREC’14  Track:  Large  Scale 
Comprehensive3D  Shape  Retrieval, 

Eurographics  Workshop  on  3D  Qbject  Retrieval  2014.  06-APR-14,  .  :  , 

09/01/2014  58.00  X.  Sun,  P.  L.  Rosin,  R.  R.  Martin,  Z.  Cheng,  Z.  Lian,  D.  Pickup,  M.  Aono,  A.  Ben  Hamza,  A.  Bronstein,  M. 

Bronstein,  S.  Bu,  U.  Castellani,  S.  Cheng,  V.  Garro,  A.  Giachetti,  A.  Godil,  J.  Han,  H.  Johan,  L.  Lai,  B.  Li, 
C.  Li,  H.  Li,  R.  Litman,  X.  Liu,  Z.  Liu,  Y.  Lu,  A.  Tatsuma,  J.  Ye.  SHREC’14  Track:Shape  Retrieval  of  Non- 
Rigid  3D  Human  Models, 

Eurographics  Workshop  on  3D  Qbject  Retrieval  2014.  06-APR-14,  .  :  , 

09/01/2014  59.00  Bingbing  Ni,  Yang  Zhou,  Shuicheng  Yan,  Pierre  Moulin,  Qi  Tian.  Pipelining  Localized  Semantic  Features 
for  Fine-Grained  Action  Recognition, 

European  Conference  on  Computer  Vision.  16-SEP-14,  .  :  , 

09/01/2014  60.00  Lingxi  Xie,  Qi  Tian,  Bo  Zhang.  Max-SIFT:  Flipping  Invariant  Descriptors  for  Web  Logo  Search, 

IEEE  International  Conference  on  Image  Processing.  27-QCT-14,  .  :  , 


09/01/2014  61.00  Liang  Zheng,  Shengjin  Wang,  Wengang  Zhou,  Qi  Tian.  Bayes  Merging  of  Muitipie  Vocabuiaries  for 
Scaiabie  Image  Retrievai, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  17-JUN-14,  .  :  , 

09/01/2014  62.00  Liang  Zheng,  Shengjin  Wang,  Ziqiong  Liu,  Qi  Tian.  Packing  and  Padding:  Coupled  Multi-index  for 
Accurate  Image  Retrieval, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  17-JUN-14,  .  :  , 

09/01/2014  63.00  Lingxi  Xie,  Jingdong  Wang,  Baining  Guo,  Bo  Zhang,  Qi  Tian.  Qrientational  Pyramid  Matching  for 
Recognizing  Indoor  Scenes, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  17-JUN-14,  .  :  , 

09/01/2014  64.00  Zhenxing  Niu,  Gang  Hua,  Xinbo  Gao,  Qi  Tian.  Semi-supervised  Relational  Topic  Model  for  Weakly 
Annotated  Image  Recognition  in  Social  Media, 

IEEE  International  Conference  on  Computer  Vision  and  Pattern  Recognition.  17-JUN-14,  .  :  , 

09/01/2014  65.00  Junjie  Cai,  Qiong  Liu,  Francine  Chen,  Dhiraj  Joshi,  Qi  Tian.  Scalable  Image  Search  with  Multiple  Index 
Tables, 

ACM  International  Conference  on  Multimedia  Retrieval.  01-APR-14,  .  :  , 

09/01/2014  66.00  Shiliang  Zhang,  Ming  Yang,  Xiaoyu  Wang,  Yuanqing  Lin,  Qi  Tian.  Semantic-aware  Co-indexing  for  Near- 
Duplicate  Image  Retrieval, 

International  Conference  on  Computer  Vision.  03-DEC-13,  .  :  , 

09/01/2014  67.00  Lingxi  Xie,  Qi  Tian,  Richang  Hong,  Shuicheng  Yan,  Bo  Zhang.  Hierarchical  Part  Matching  for  Fine- 
Grained  Visual  Categorization, 

International  Conference  on  Computer  Vision.  03-DEC-14,  .  :  , 

09/01/2014  68.00  Zhendong  Mao,  Yongdong  Zhang,  Qi  Tian.  A  Novel  Feature  Descriptor  Exploring  Anisotropy  and  Non¬ 
uniformity, 

Pacific-rim  Conference  on  Multimedia.  13-DEC-13,  .  :  , 


TOTAL: 


67 


Number  of  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


(d)  Manuscripts 


Received  Paper 


08/30/2012  2.00 

08/30/2012  4.00 

08/30/2012  3.00 

08/31/2015  94.00 

08/31/2015  93.00 

08/31/2015  92.00 

09/01/2014  51.00 

09/01/2014  50.00 

09/01/2014  52.00 

09/01/2014  53.00 


Xinmei  Tian,  Yijuan  Lu.  Discriminative  Codebook  Learning  for  Web  Image  Search, 

Signal  Processing  (04  2012) 

Wengang  Zhou,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  Principal  Visual  Word  Discovery  for  AutomaticLicense 
Plate  Detection, 

Transactions  on  Image  Processing  (04  2012) 

Wengang  Zhou,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  SIFT  Match  Verification  by  Geometric  Coding  for  Large 
Scale  Partial-duplicate  Web  Image  Search, 

ACM  Transactions  on  Multimedia  Computing,  Communications  and  Applications  (01  2012) 

D.  Pickup,  X.  Sun,  P.  L.  Rosin,  R.  R.  Martin,  Z.  Cheng,  Z.  Lian,  M.  Aono,  A.  Ben  Hamza,  A.  Bronstein,  M. 
Bronstein,  S.  Bu,  U.  Castellani,  S.  Cheng,  V.  Garro  ,  A.  Giachetti,  A.  Godil,  L.  Isaia  ,  J.  Han  ■  H.  Johan  ■  L. 
Lai  ■  B.  Li  ■  C.  Li  ■  H.  Li  ■  R.  Litman  ■  X.,  Liu  ■  Z.  Liu  ■  Y.  Lu  ■  G.  Tam  ■  A.  Tatsuma  ■  J.  Ye  ■.  Shape 
Retrieval  of  Non-Rigid  3D  Human  Models, 

International  Journal  of  Computer  Vision  (09  2014) 

Bo  Li,  Yijuan  Lu,  Henry  Johan,  Ribel  Fares.  Sketch-based  3D  model  retrieval  utilizing  adaptive  view 
clustering  and  semantic  information. 

Computer  Vision  and  Image  Understanding  (05  2015) 

Bo  Li,  Henry  Johan,  Yuxiang  Ye,  Yijuan  Lu.  Efficient  3D  Reflection  Symmetry  Detection:  a  View-Based 
Approach, 

Graphical  Models  (03  2015) 

Xinmei  Tian,  Linjun  Yang,  Yijuan  Lu,  Qi  Tian.  Topic-aware  Image  Search  Reranking, 

IEEE  TRANSACTIQNS  QN  Systems,  Man,  Cybernetics  (11  2012) 

Bo  Li,  Yijuan  Lu,  Henry  Johan,  Ribel  Fares.  Sketch-based  3D  model  retrieval  utilizing  adaptive 
viewclustering  and  semantic  information. 

Pattern  Recognition  (05  2014) 

Wengang  Zhou,  Houqiang  Li,  Yijuan  Lu,  Qi  Tian.  BSIFT:  towards  Data-Independent  Codebook  forLarge 
Scale  Image  Search, 

IEEE  TRANSACTIQNS  QN  Image  Processing  (09  2013) 

Bo  Li,  Yijuan  Lu,  Chunyuan  Li,  Afzal  Godil,  Tobias  Schreck,  Masaki  Aono,  Martin  Burtscher,  Qiang  Chen, 
Nihad  Karim  Chowdhury,  Bin  Fang,  Hongbo  Fu,  Takahiko  Furuya,  Haisheng  Li,  Jianzhuang  Liu,  Henry 
Johan,  Ryuichi  Kosaka,  Hitoshi  Koyanagi,  Ryutarou  Qhbuchi,  Atsushi  Tatsuma,  Yajuan  Wan,  Chaoli 
Zhang,  Changqing  Zou.  A  comparison  of  3D  shape  retrieval  methods  based  on  a  large-scale  benchmark 
supportingmultimodal  queries. 

Computer  Vision  and  Image  Understanding  (06  2014) 


TOTAL: 


10 


Number  of  Manuscripts: 


Books 


Received  Book 


TOTAL: 


Received  Book  Chapter 


TOTAL: 


Patents  Submitted 


Patents  Awarded 


Awards 


Award: 

This  project  has  resulted  in  the  folio  wing”'”'  prestigious  awards: 

T  Best  Paper  Award,  the  4th  ACM  International  Conference  on  Internet  Multimedia  Computing  and  Service  (ICIMCS 
21012),  September  2012. 

•  Best  Paper  Award,  the  IEEE  International  Conference  on  Multimedia  and  Expo  (ICME)  2013. 

•  Best  Paper  Award,  the  Pacific-rim  Conference  on  Multimedia  (PCM),  2013. 

•  Best  Paper  Award,  the  19th  International  Conference  on  Multimedia  Modeling  (MMM),  January  2013. 

•  Best  Paper  Award,  the  ACM  International  Conference  on  Multimedia  Retrieval  (ICMR),  June,  2015. 

•  Best  Student  Paper  Candidate,  the  IEEE  International  Conference  on  Multimedia  and  Expo  (ICME),  July  28-July  2, 
2015. 

Honor  and  Recognition: 

Faculty: 

•  Dr.  Yijuan  Lu  (PI)  was  awarded  Texas  State  University  Junior  Faculty  Research  Enhancement  Award  2012. 

•  Dr.  Yijuan  Lu  (PI)  was  nominated  to  Texas  State  University  Presidential  Award  For  Excellence  In  Scholarly/Creative 

Activities  2012. 

•  Dr.  Qi  Tian  (co-PI)  was  promoted  to  Full  Professor  at  UTSA,  December  2012. 

•  Dr.  Yijuan  Lu  (PI)  and  her  team  won  the  First  Place  in  “Large  Scale  Sketch-Based  3D  Shape  Retrieval  Competition”, 
First  Place  in  “Large-Scale  Partial  Shape  Retrieval  Using  Simulated  Range  Images  Competition”,  and  Second  Place  in 
“Retrieval  of  Objects  Captured  with  Low-Cost  Depth-Sensing  Cameras  Competition”  in  EUROGRAPHICS  3D  Shape 
Retrieval  Contest  2013  (SHREC’13) 

•  Dr.  Yijuan  Lu  (PI)  is  nominated  to  Texas  State  University  Presidential  Award  For  Excellence  In  Scholarly/Creative 
Activities  2013. 

•  Dr.  Yijuan  Lu  (PI)  received  College  Achievement  Award  for  the  Presidential  Award  for  Excellence  in 
Scholarly/Creative  Activities,  College  of  Science  and  Engineering,  Texas  State  University,  2014. 

•  Dr.  Qi  Tian  (co-PI)  received  Research  Achievement  Award,  College  of  Science,  UTSA,  December  2014. 

•  Dr.  Yijuan  Lu  (PI)  received  Presidential  Distinction  Award  for  Excellence  in  Service,  Texas  State  University,  2015. 

•  Dr.  Yijuan  Lu  (PI)  and  her  team  won  the  First  Place  in  “3D  Sketch-Based  3D  Shape  Retrieval  Competition”  in 
EUROGRAPHICS  3D  Shape  Retrieval  Contest  2016  (SHREC’16). 

•  Dr.  Qi  Tian  (co-PI)  is  elevated  to  IEEE  Fellow  (class  2016),  November  2015. 

Student: 

•  Shaomin  Fang  received  Excellence  in  Graduate  Research  Award  at  Texas  State  University  2013. 

•  Yuxiang  Ye  received  Excellence  in  Graduate  Research  Award  at  Texas  State  University  2014,  2015. 

•  Junjie  Cai  received  ACM  Multimedia  Student  Travel  Award  2014. 

•  Junjie  Cai  received  UTSA  Graduate  Student  Professional  Development  Award  2014. 


Graduate  Students 


NAME 

PERCENT  SUPPORTED  Discipline 

Shaomin  Fang 

0.50 

Travis  Bulgerin 

0.10 

Yuxiang  Ye 

0.50 

jie  Xiao 

0.08 

Junjie  Cai 

1.00 

Xia  Li 

0.08 

Yang  Zhou 

0.02 

FTE  Equivalent: 

2.28 

Total  Number: 

7 

Names  of  Post  Doctorates 


NAME 

PERCENT  SUPPORTED 

Xinmei  Tian 

0.10 

Wengang  Zhou 

0.10 

Bo  Li 

1.00 

FTE  Equivalent: 

1.20 

Total  Number: 

3 

Names  of  Faculty  Supported 


NAME 

PERCENT  SUPPORTED 

National  Academy  Member 

Yijuan  Lu 

0.08 

No 

Qi  Tian 

0.08 

No 

FTE  Equivalent: 

0.16 

Total  Number: 

2 

Names  of  Under  Graduate  students  supported 


NAME 

PERCENT  SUPPORTED 

Discipline 

Daniel  Brooks 

0.08 

Computer  and  Computational  Sciences 

Robert  Dunk 

0.08 

Computer  and  Computational  Sciences 

FTE  Equivalent: 

0.16 

Total  Number: 

2 

Student  Metrics 

This  section  only  applies  to  graduating  undergraduates  supported  by  this  agreement  in  this  reporting  period 

The  number  of  undergraduates  funded  by  this  agreement  who  graduated  during  this  period: . 2.00 

The  number  of  undergraduates  funded  by  this  agreement  who  graduated  during  this  period  with  a  degree  in 

science,  mathematics,  engineering,  or  technology  fields: . 2.00 

The  number  of  undergraduates  funded  by  your  agreement  who  graduated  during  this  period  and  will  continue 

to  pursue  a  graduate  or  Ph.D.  degree  in  science,  mathematics,  engineering,  or  technology  fields: . 0.00 

Number  of  graduating  undergraduates  who  achieved  a  3.5  GPA  to  4.0  (4.0  max  scale): . 2.00 

Number  of  graduating  undergraduates  funded  by  a  DoD  funded  Center  of  Excellence  grant  for 

Education,  Research  and  Engineering: . o.OO 

The  number  of  undergraduates  funded  by  your  agreement  who  graduated  during  this  period  and  intend  to  work 

for  the  Department  of  Defense . 0.00 

The  number  of  undergraduates  funded  by  your  agreement  who  graduated  during  this  period  and  will  receive 

scholarships  or  fellowships  for  further  studies  in  science,  mathematics,  engineering  or  technology  fields: . 0.00 

Names  of  Personnel  receiving  masters  degrees 

NAME 
Shaomin  Fang 
Travis  Bufgerin 
Yuxiang  Ye 

Total  Number:  3 


Names  of  personnel  receiving  PHDs 


NAME 

xia  Li 

Jie  Xiao 

junjie  Cai 

Total  Number: 

3 

Names  of  other  research  staff 


NAME 

PERCENT  SUPPORTED 

FTE  Equivalent: 

Total  Number: 

Sub  Contractors  (DD882) 


Inventions  (DD882) 


Scientific  Progress 


1 )  Foreword 

This  project  started  in  January  2012  and  ended  in  January  2016  with  one  year  no-cost  extension  from  January  2015  to  January 
2016.  There  are  three  Post-docs  (Xinmei  Tian,  Wengang  Zhou,  Bo  Li),  three  Ph.D.  students  (Jie  Xiao,  2008-2014,  Xia  Li,  2008- 
2014,  Junjie  Cai,  2011-2015,  Yang  Zhou,  2015),  and  two  master  students  (Shaomin  Fang,  2012-2013,  Yuxiang  Ye,  2014- 
2016)  supported  by  this  ARO  grant.  Xinmei  Tian  and  Wengang  Zhou  joined  CS  Department  of  University  of  Science  and 
Technology  of  China  as  tenure-track  assistant  professors.  Bo  Li  joined  Department  of  Mathematics  and  Computer  Science  in 
University  of  Central  Missouri  as  an  assistant  professor  in  2015.  Xia  Li  joined  in  Google  in  2014  as  a  Software  Development 
Engineer  (SDE).  Jie  Xiao  joined  in  Rocket  Fuel  Inc.  as  a  Research  Scientist  in  2014.  Junjie  Cai  joined  in  Blippar  Inc.  as  a 
Research  Scientist  in  2015.  Shaomin  Fang  joined  in  Emerson  at  Austin  in  2013  and  Yuxiang  Ye  will  graduate  in  May  2016  and 
will  join  in  Google  June  2016.  The  project  has  been  finished  successfully  within  the  timeline  and  all  the  research  goals  proposed 
in  the  original  proposal  have  been  achieved. 

During  the  project  period,  the  PI  Dr.  Lu  was  promoted  to  Associate  Professor  at  Texas  State.  Dr.  Tian  was  promoted  to  Full 
Professor  at  UTSA  and  was  elevated  to  IEEE  Fellow  (class  2016).  The  Pis  sincerely  thank  for  ARO  and  the  program  manager 
Dr.  Liyi  Dai  for  supporting  this  research  during  the  past  4  years.  Without  the  dedicated  support  from  Dr.  Dai  and  ARO,  this 
project  won’t  be  done  successfully.  We  look  forwards  to  the  future  research  collaboration  with  Dr.  Dai  and  ARO. 

2.  Statement  of  the  problem  studied 

The  explosive  growth  of  Internet  Media  (partial-duplicate/similar  images,  3D  objects,  3D  models,  etc.)  sheds  bright  light  on 
many  promising  applications  in  forensics,  surveillance,  3D  animation,  mobile  visual  search,  and  3D  model/object  search. 
Compared  with  the  general  images,  partial-duplicate  images  have  some  intrinsic  properties  such  as  high  repeatability  of  local 
features,  consistent  local  patch  appearance,  and  stable  spatial  configuration.  Compared  with  the  general  2D  objects,  3D 
models/objects  consist  of  3D  data  information  (typically  a  list  of  vertices  and  faces)  to  represent  3D  objects.  However,  these 
unique  properties  of  partial-duplicate  images  and  3D  models  have  not  been  well  exploited  to  design  effective  and  efficient 
search  algorithms.  Because  of  this,  existing  works  for  large-scale  partial-duplicate  image  retrieval  and  3D  model  retrieval  suffer 
from  two  major  problems:  low  accuracy  and  low  efficiency.  These  problems  make  them  fall  far  below  many  applications’ 
requirement.  This  project  has  investigated  many  key  problems  in  large-scale  partial-duplicate/similar  image  and  3D  model 
retrieval:  feature  descriptor  problem,  image  representation  problem,  index  strategy  problem,  feature  quantization  problem, 
image  search  results  quality  assessment  problem,  image  search  reranking  problem,  sketch-based  3D  model  retrieval  problem, 
and  related  search  problems  and  has  proposed  a  series  of  effective  and  efficient  approaches  to  solve  them. 

3.  Summary  of  the  scientific  accomplishments 

The  key  elements  developed  in  this  project  will  make  significant  contributions  to  improve  the  performance  of  large  scale  partial 
duplicate  image  retrieval  system,  large  scale  3D  model  retrieval  system,  and  personalized  image  retrieval.  The  research  teams 
are  one  of  the  first  (to  their  best  knowledge)  in  the  field  who 

•  propose  a  novel  strategy  to  generate  a  reliable,  easily  updated  visual  dictionary  with  low  computational  cost.  In  our  method, 
no  visual  codebook  is  needed  to  be  trained  and  the  quantizer  is  independent  of  collections  of  images; 

•  propose  to  formulate  the  codebook  construction  and  contextual  subspace  learning  into  one  optimization  problem  and 
embed  semantic  information; 

•  apply  visual  word  expansion  approach  to  reduce  the  feature  quantization  error  and  boost  the  retrieval  recall; 

•  propose  to  embed  the  spatial  information  between  local  features  into  the  inverted  index  file  efficiently; 

•  propose  to  build  geometric  square  coding  and  geometric  fan  coding  together  to  fully  capture  the  global  geometric  context  of 
local  features  in  an  image  and  effectively  discover  false  feature  matches  between  images. 

•  propose  a  novel  flexible  scale  invariant  feature  transform  (SIFT)  binarization  (FSB)  algorithm; 

•  propose  to  explore  the  use  of  partial  binary  descriptors  as  direct  codebook  indices  (addresses); 

•  introduce  a  novel  IDF  family  by  the  use  of  Lp-norm  pooling  technique; 

•  propose  a  novel  attribute-assisted  retrieval  model  for  reranking  images; 

•  build  the  largest  comprehensive  3D  shape  database  that  contains  different  types  of  models  including  generic,  articulated, 
CAD  and  architecture  models; 

•  construct  the  largest  sketch-3D  model  benchmark  that  contains  a  large  number  of  diverse  types  of  sketches  and  models; 

•  apply,  compare,  and  analyze  state-of-the-art  3D  model  retrieval  approaches  on  the  collected  sketch  and  3D  model 
benchmark; 

•  propose  a  simple  and  efficient  view-based  3D  symmetry  detection  method. 

•  propose  a  novel  matching  verification  scheme  based  on  binary  SIFT  (BSIFT); 

•  propose  two  novel  binary  feature  descriptors:  COGE  and  Edge-SIFT  and  investigate  representing  the  spatial  context  of 
local  features  into  binary  codes; 

•  encode  local  invariant  features  and  high  level  semantic  attributes  together  to  effectively  enhance  the  discriminative 
capability  of  inverted  indexes; 

•  use  the  semantic  classification  information  to  reduce  the  semantic  gap,  as  well  as  to  adequately  utilize  the  better¬ 
performing  global  feature  matching  to  improve  sketch-based  3D  model  retrieval  efficiency; 

•  propose  a  3D  visual  complexity  metric  based  on  the  viewpoint  entropy  distribution  of  a  set  of  sample  views; 

•  explore  the  correlations  between  users’  different  touch  behaviors  and  human  attention,  investigate  their  contribution  to  the 
human  eye  fixation,  and  propose  a  novel  supervised  learning  approach  to  learn  good  image  saliency  maps  from  multiple  touch 


behaviors. 

During  the  past  year  (2012),  the  foiiowing  scientific  accompiishments  have  been  made: 

1)  Scaiar  Quantization 

The  research  team  investigate  bit  vector  quantization  aigorithms  to  address  the  “high  computationai  cost”,  ‘limited  reiiabiiity”, 
and  “update  inefficiency”  probiems  in  the  codebook  generation  stage.  A  novei  feature  quantization  aigorithm,  scaiar 
quantization,  is  proposed.  With  scaiar  quantization,  a  iocai  feature  is  quantized  to  a  descriptive  and  discriminative  bit-vector,  of 
which  the  first  tens  of  bits  are  taken  out  as  code  word.  Our  quantizer  is  independent  of  coiiections  of  images.  In  addition,  the 
result  of  scalar  quantization  naturally  lends  itself  to  adapt  to  the  classic  inverted  file  structure  for  image  indexing.  Moreover,  the 
quantization  error  can  be  flexibly  reduced  and  controlled  by  efficiently  enumerating  nearest  neighbors  of  code  words. 

2)  Discriminative  Codebook  Learning 

The  research  team  propose  a  novel  supervised  discriminative  codebook  learning  method,  which  can  find  a  contextual  subspace 
to  embed  the  semantic  information  into  codebook  generation  and  learn  a  contextual  subspace  and  discriminative  codebook 
simultaneously.  In  the  learned  new  space,  images  from  different  classes  can  be  well  separated  and  images  from  the  same 
class  are  close  to  each  other.  The  research  team  applied  the  proposed  method  on  Web  image  search  re-ranking  problem  and 
the  experimental  results  on  two  real  Web  image  search  datasets  have  demonstrated  the  effectiveness  of  our  approach  and  its 
superiority  than  other  state-of-the-art  codebook  learning  methods. 

3)  Visual  Word  Expansion  and  Binary  SIFT 

The  research  team  propose  a  novel  visual  word  expansion  approach  to  improve  the  feature  quantization  accuracy  and  boost 
the  retrieval  recall.  The  proposed  visual  word  expansion  scheme  is  based  on  the  observation  that  the  expected  nearest  visual 
word  to  a  test  feature  is  always  close  to  the  approximate  nearest  visual  word,  which  can  be  efficiently  identified  by  the 
hierarchical  k-NN  search.  Experiments  on  image  search  in  million-scale  dataset  demonstrate  the  effectiveness  of  the  proposed 
visual  word  expansion  approach. 

Moreover,  the  research  team  design  a  new  scheme  to  transform  a  SIFT  descriptor  to  a  binary  bit  stream,  called  binary  SIFT. 
Extensive  study  with  large-scale  (trillion)  samples  reveal  that  the  generated  binary  SIFT  effectively  keeps  the  distance  metric  of 
the  original  SIFT  descriptor. 

4)  Embedding  Spatial  Context  into  Inverted  File 

The  research  team  explore  two  novel  ways  to  embed  the  spatial  information  between  local  features  into  the  inverted  index  file. 

A  one-one  relationship  approach  is  proposed  to  record  feature  tuples  into  the  inverted  file,  which  consist  of  a  pair  of  single 
features  and  their  corresponding  spatial  relationship.  A  one-multiple  relationship  approach  is  proposed  to  record  the  spatial 
relationship  between  a  single  feature  and  its  surrounding  features,  which  are  clustered  into  different  groups  based  on  their 
locations.  With  the  embedded  spatial  information  in  the  inverted  index  file,  feature  matching  with  geometric  verification  can  be 
performed  implicitly,  efficiently,  and  effectively  in  the  inverted  file  traversing  step,  which  make  the  proposed  methods  more 
suitable  for  large-scale  image  search. 

5)  Spatial  and  Geometric  Coding  for  Geometric  Verification 

The  research  team  investigate  two  different  coding  algorithms  “Spatial  Coding”  and  “Geometric  Coding”  to  achieve  fast  and 
accurate  geometric  verification  of  local  features  in  the  post  processing  step.  The  proposed  spatial  coding  algorithm  encodes 
the  relative  positions  between  each  pair  of  features  along  the  horizontal  (X-axis)  and  vertical  (Y-axis)  direction  by  generating 
two  binary  spatial  maps  X-map  and  Y-map.  The  proposed  geometric  coding  method  consists  of  geometric  square  coding  and 
geometric  fan  coding,  which  describe  the  spatial  relationships  of  local  features  (e.g.  inside  or  outside  of  geometric  square  and 
fan)  into  H-map,  V-map,  and  S-map  for  global  verification  to  remove  geometrically  inconsistent  feature  matches.  These  two 
approaches  are  not  only  computationally  efficient,  but  also  effective  in  detecting  partial-duplicate  images  with  rotation,  scale 
changes,  partial-occlusion,  and  background  clutter. 

6)  Applications 

During  the  exploration  of  solutions  to  solve  the  above  scientific  barriers  in  the  partial  duplicate  image  retrieval,  the  research 
team  find  some  other  interesting  applications,  which  may  share  similar  problems,  such  as  license  plate  detection  (characters  in 
different  license  plates  are  duplicates  of  each  other),  similar  song  retrieval,  and  3D  image  search.  Motivated  by  these 
similarities,  several  new  approaches  are  proposed  for  these  applications  by  bringing  the  ideas  in  the  partial  duplication  image 
retrieval. 

Observing  that  characters  in  different  license  plates  are  duplicates  of  each  other,  we  bring  in  the  idea  of  bag-of-words  (BoW) 
model  popularly  into  license  plate  detection.  The  research  team  propose  a  novel  scheme  to  automatically  locate  license  plate 
by  principal  visual  word  discovery  and  local  feature  matching.  For  large  scale  similar  song  retrieval,  the  research  team  proposed 
to  utilize  beat-aligned  chroma  patches  and  applied  location  coding  scheme  to  encode  the  location  relationships  among  beat- 
aligned  chroma  patches  in  a  song.  Our  approach  is  both  efficient  and  effective  to  discover  true  matches  of  beat  chroma  patches 
between  songs  with  low  computational  cost.  In  the  3D  image  retrieval,  the  research  team  propose  a  sketch-based  3D  model 
retrieval  algorithm  by  utilizing  viewpoint  entropy-based  adaptive  view  clustering  and  shape  context  matching.  The  algorithm  is 
tested  on  a  latest  sketch-based  3D  model  retrieval  benchmark  and  the  results  demonstrate  the  superior  performances  and 
advantages  of  our  algorithm. 

During  the  past  year  (2013),  the  following  scientific  accomplishments  have  been  made: 

7)  Feature  Quantization 

The  research  team  propose  to  construct  a  supporting  visual  word  table  for  all  visual  words  by  visual  word  expansion.  Given  the 
initial  quantization  result,  multiple  approximate  nearest  visual  words  are  identified  by  checking  supporting  visual  word  table, 
which  benefits  the  retrieval  recall.  Moreover,  a  matching  verification  scheme  based  on  binary  SIFT  (BSIFT)  is  presented.  The 
L2 -distance  between  original  SIFT  descriptors  is  demonstrated  to  be  well  kept  with  the  metric  of  Hamming  distance  betThe 


research  teamen  the  corresponding  binary  SIFT  vectors.  With  the  BSIFT  verification,  false  positive  matches  can  be  effectively 
and  efficiently  identified  and  removed,  which  greatly  improves  the  accuracy  of  large-scale  image  search.  The  proposed 
approach  is  evaluated  by  conducting  partial-duplicate  image  search  on  a  one-million  image  database.  The  experimental  results 
demonstrate  the  effectiveness  and  efficiency  of  the  proposed  scheme. 

8)  Binary  Feature  Descriptors  for  Image  Retrieval 

The  research  team  propose  two  novel  binary  feature  descriptors:  COGE  and  Edge-SIFT.  The  proposed  COGE  exploits  the 
anisotropy  and  the  non-uniformity  of  the  underlying  gradient  distributions.  Therefore,  it  exhibits  better  results  than  state-of-the- 
art  methods  with  respect  to  distinctiveness,  robustness,  and  storage  cost.  The  proposed  Edge-SIFT  is  generated  from  the 
binary  edge  maps  of  scale  and  orientation-normalized  image  patches.  By  preserving  both  locations  and  orientations  of  edges 
and  compressing  the  sparse  binary  edge  maps  with  a  boosting  strategy,  Edge-SIFT  shows  strong  discriminative  power  with 
compact  representation. 

In  addition,  the  research  team  investigate  representing  the  spatial  context  of  local  features  into  binary  codes  to  implicitly  achieve 
geometric  verification  by  efficient  comparison  of  the  binary  codes.  Moreover,  a  weighted  Hamming  distance  ranking  algorithm 
(WhRank)  to  rank  the  binary  codes  of  hashing  methods  is  developed.  By  assigning  different  bit-level  weights  to  different  hash 
bits,  the  returned  binary  codes  are  ranked  at  a  finer-grained  binary  code  level.  Experiments  on  several  state-of-the-art 
benchmark  data  sets  with  million-scale  distractor  images  demonstrate  the  effectiveness  of  the  proposed  algorithms. 

9)  Semantic-aware  Co-indexing  for  Image  Search 

The  research  team  propose  a  novel  approach  for  embedding  semantic  attributes  into  the  pipeline  of  image  search.  Inverted 
indexes  in  image  retrieval  not  only  allow  fast  access  to  database  images  but  also  summarize  all  knowledge  about  the  database, 
so  that  their  discriminative  capacity  largely  determines  the  retrieval  performance.  For  vocabulary  tree  based  image  retrieval,  a 
semantic-aware  co-indexing  algorithm  is  proposed  to  jointly  embed  two  strong  cues  into  the  inverted  indexes:  1)  local  invariant 
features  that  are  robust  to  delineate  low-level  image  contents,  and  2)  semantic  attributes  from  large-scale  object  recognition 
that  may  reveal  image  semantic  meanings.  Encoding  these  two  distinct  cues  together  effectively  enhances  the  discriminative 
capability  of  inverted  indexes.  Such  co-indexing  operations  are  totally  off-line,  and  therefore  introduce  small  computation 
overhead  to  online  query.  Experiments  and  comparisons  with  recent  retrieval  methods  have  evidently  demonstrated  the 
significant  improvement  in  retrieval  performance  and  efficiency. 

10)  Sketch-based  Image/SD  Model  Retrieval 

The  research  team  propose  a  new  sketch  feature  to  capture  both  the  local  and  global  information  of  a  sketch.  The  proposed 
sketch  feature  can  handle  the  rotation  of  sketches  better  by  further  integrating  a  set  of  rotation-invariant  global  features  for  a 
sketch.  In  addition,  an  intelligent  sketch  recognizer  through  supervised  learning  is  developed  to  correctly  capture  the  semantic 
meanings  of  users’  sketches.  The  research  team  also  conducts  a  comprehensive  study  of  the  semantic  gap  between  user 
sketches  and  3D  models  and  proposes  a  novel  semantic  sketch-based  3D  model  search  algorithm  to  bridge  such  a  semantic 
gap.  The  experimental  results  confirmed  that  the  proposed  sketch  recognizer  is  more  robust  to  sketch  rotation  and  can 
describe  user  sketches  well.  In  addition,  the  proposed  semantic  retrieval  approach  significantly  improves  the  retrieval  accuracy 
and  reduces  the  search  time  over  other  state-of-the-art  sketch-based  3D  model  retrieval  algorithms.  It  further  validated  that  the 
proposed  approach  could  bridge  the  semantic  gap  between  the  diverse  query  sketches  and  3D  models  effectively. 

1 1 )  Large  Scale  3D  Model  Retrieval 

The  research  team  propose  a  new  3D  model  retrieval  algorithm  by  adaptive  view  clustering.  A  novel  3D  visual  complexity  metric 
is  formulated  based  on  the  viewpoint  entropy  distribution  of  a  set  of  uniformly  sampled  views  of  the  3D  model.  Then,  it  guides 
adaptive  view  clustering  of  a  3D  model  to  shortlist  a  set  of  representative  sample  views  for  2D-3D  comparison,  which  largely 
reduces  the  number  of  sample  views  for  comparison  and  therefore  greatly  improves  the  computational  efficiency.  After  that,  a 
shape  context  matching  algorithm  is  used  for  the  2D-3D  matching  between  the  query  and  the  representative  views  for  each 
model.  Experimental  results  on  several  latest  benchmarks  have  evidently  demonstrated  our  significant  improvement  in  retrieval 
performance  and  efficiency. 

In  addition,  the  research  team  organize  a  “Large  Scale  Sketch-based  3D  Model  Retrieval”  competition  in  the  Shape  Retrieval 
Contest  (SHREC)  2013,  which  is  held  at  the  3D  Object  Retrieval  (3DOR)  workshop  of  Eurographics  2013.  In  the  meantime,  the 
research  team  also  attend  two  SHREC  contests  on  “Large-Scale  3D  Partial  Shape  Retrieval  Track  Using  Simulated  Range 
Images”  and  “Retrieval  of  3D  Objects  Captured  with  Low-Cost  Depth-Sensing  Cameras”.  In  these  competitions,  the  proposed 
3D  model  retrieval  algorithm  receives  First  Place  in  the  large-scale  sketch-based  3D  retrieval  track.  First  Place  in  the  range 
scan  track,  and  obtains  Second  Place  in  the  low-cost  depth-sensing  camera  track. 

12)  Touch-based  Personalized  Image  Search 

The  research  team  quantitatively  studied  and  analyzed  human  attention  from  a  variety  of  touch  behaviors,  and  proposes  a  set 
of  valuable  features  from  the  touch  information.  A  series  of  experiments  are  designed  and  conducted  with  the  conventional  eye- 
fixation  based  saliency  serving  as  the  ground  truth.  An  image  browsing  app  is  designed  on  a  touch  mobile  phone  to  collect 
users’  touch  behavior  data.  In  addition,  a  novel  touch  saliency  learning  approach  is  proposed  to  automatically  learn  the 
correlation  between  different  touch  behaviors  and  human  eye  fixations,  and  then  to  derive  a  good  image  saliency  map  from  a 
variety  of  touch  behaviors.  During  the  process  of  building  a  supervised  learning  model,  the  weights  of  different  human  touch 
behaviors  are  learned,  which  indicate  the  different  contributions  of  these  behaviors  to  the  user’s  attention  information.  The 
experimental  results  have  demonstrated  the  validity  of  the  study  and  the  potential  and  effectiveness  of  the  proposed  approach. 

13)  Applications 

In  addition,  the  research  team  explored  many  important  related  problems:  1)  “human  movement  summarization  and  depiction 
from  videos”  problem.  Previous  action  summarization  methods  all  rely  on  3D  motion  capture  or  manually  labeled  data,  without 
which  depicting  actions  is  a  challenging  task.  A  novel  scheme  is  proposed  to  automatically  summarize  and  depict  human 


movements  from  2D  videos  without  3D  motion  capture  or  manually  labeled  data.  The  proposed  method  can  successfully 
segment  videos  into  sub-actions  with  an  effective  streamline  matching  scheme,  track  points  on  body  parts,  and  depict  the 
human  articulated  motion  with  arrows  and  motion  particles.  2)  “image  matching”  problem.  A  new  matching  method,  Semantic- 
Spatial  Matching  (SSM)  is  proposed.  SSM  conducts  region  matching  by  considering  both  the  spatial  layout  and  the  semantic 
content  information.  SSM  has  the  advantage  not  only  being  robust  to  rotation,  flipping  and  other  variances,  but  also  simple  and 
easy  for  implementation.  Experiments  on  two  benchmark  datasets  demonstrate  its  effectiveness  in  object  and  scene 
classifications.  3)  “3D  scene  reconstruction”  problem.  The  performance  of  the  PMVS  (Patch-based  Multi-View  Stereo  software) 
for  scene  reconstruction  from  stereo  pairs  of  scenes  captured  by  a  simple  3D  camera  has  been  analyzed.  A  Canny  feature- 
based  PMVS  algorithm  and  a  preliminary  disparity  map-based  stereo  reconstruction  algorithm  are  proposed.  Both  proposed 
approaches  are  promising  for  related  applications  which  require  effective  3D  scene  reconstruction  from  a  set  of  sparsely 
sampled  pairs.  4)  “image  annotation”  problem.  An  automatic  image  annotation  approach  using  semantic  relevance  is  proposed. 
It  constructs  an  improved  probabilistic  model  to  characterize  different  regions’  contributions  to  the  semantics  more  accurately 
based  on  the  spatial  ,  visual  and  contextual  information  of  the  region.  And  it  also  helps  expand  the  coverage  of  the  semantic 
concept  with  semantic  relevance  information. 

During  the  past  year  (2014),  the  following  scientific  accomplishments  have  been  made: 

14)  Compact  Binary  Feature  Descriptors  for  Image  Retrieval 

The  research  team  propose  a  novel  flexible  scale  invariant  feature  transform  (SIFT)  binarization  (FSB)  algorithm  for  large-scale 
image  search.  The  FSB  algorithm  explores  the  magnitude  patterns  of  SIFT  descriptor.  It  is  unsupervised  and  the  generated 
binary  codes  are  demonstrated  to  be  dispreserving.  Besides,  a  new  searching  strategy  is  investigated  to  find  target  features 
based  on  the  cross-indexing  in  the  binary  SIFT  space  and  original  SIFT  space.  The  experiments  on  large-scale  partial  duplicate 
image  retrieval  system  demonstrate  the  effectiveness  and  efficiency  of  the  proposed  algorithm. 

In  addition,  the  research  team  explored  the  use  of  partial  binary  descriptors  as  direct  codebook  indices  (addresses).  Typically, 
binary  descriptors  are  clustered  into  codewords  and  quantized  with  Hamming  distance,  following  the  conventional  bag-of-words 
strategy.  However,  such  codewords  formulated  in  Hamming  space  do  not  present  obvious  indexing  and  search  performance 
improvement  as  compared  to  the  Euclidean  codewords.  Without  explicit  codeword  construction,  the  proposed  approach 
performs  to  build  multiple  index  tables  which  concurrently  check  for  collision  of  the  same  hash  values.  The  evaluation  is 
performed  on  two  public  image  datasets  and  the  experimental  results  demonstrate  the  indexing  efficiency  and  retrieval 
accuracy  of  our  approach. 

15)  TF-IDF  strategy  for  Image  Retrieval 

The  research  team  introduced  a  novel  IDF  family  by  the  use  of  Lp-norm  pooling  technique.  Carefully  designed,  the  proposed 
IDF  considers  the  term  frequency,  document  frequency,  the  complexity  of  images,  as  well  as  the  codebook  information.  The 
research  team  further  proposes  a  parameter  tuning  strategy,  which  helps  to  produce  optimal  balancing  between  TF  and  pIDF 
weights,  yielding  the  so-called  Lp-norm  IDF  (pIDF).  Further,  by  counting  for  the  term-frequency  in  each  image,  the  proposed 
pIDF  helps  to  alleviate  the  visual  word  burstiness  phenomenon. 

Moreover,  the  team  initializes  to  embed  multiple  binary  features  at  indexing  level.  To  model  correlation  between  features,  a 
multi-IDF  scheme  is  introduced,  through  which  different  binary  features  are  coupled  into  the  inverted  file.  As  an  extension,  the 
research  team  also  explores  the  fusion  of  binary  color  feature  into  image  retrieval.  The  joint  integration  of  the  SIFT  visual  word 
and  binary  features  greatly  enhances  the  precision  of  visual  matching,  reducing  the  impact  of  false  positive  matches. 

16)  An  attribute-assisted  Reranking  Model  for  Web  Image  Search 

The  research  team  proposed  a  novel  attribute-assisted  retrieval  model  for  reranking  images.  Based  on  the  classifiers  for  all  the 
predefined  attributes,  each  image  is  represented  by  an  attribute  feature  consisting  of  the  responses  from  these  classifiers.  A 
hypergraph  is  used  to  model  the  relationship  between  images  by  integrating  low-level  visual  features  and  semantic  attribute 
features.  And  hypergraph  ranking  is  applied  to  re-order  the  images.  Its  basic  principle  is  that  visually  similar  images  should 
have  similar  ranking  scores.  A  visual-attribute  joint  hypergraph  learning  approach  has  been  proposed  to  simultaneously  explore 
two  information  sources.  The  extensive  experiments  have  been  conducted  on  1,000  queries  in  MSRA-MM  V2.0  dataset.  The 
experimental  results  demonstrate  the  effectiveness  of  the  proposed  attribute-assisted  Web  image  search  reranking  method. 

17)  Large  Scale  Comprehensive  3D  Model  Retrieval 

The  research  team  built  a  Large  Scale  Comprehensive  3D  model  Benchmark  dataset  (SHREC14LSGTB).  This  3D  shape 
dataset  contains  8,987  models  in  a  variety  of  types  (generic,  articulated,  CAD  and  architecture  models)  and  classified  into  171 
categories.  This  work  is  the  first  work  to  integrate  existing  3D  model  datasets  to  form  a  new,  larger  benchmark  corpus  for 
comprehensive  3D  shape  retrieval.  The  newly  created  benchmark  is  the  most  extensive  to  date  in  terms  of  the  number  of 
semantic  query  categories  covered  as  well  as  the  variations  of  model  types.  In  particular,  it  combines  generic  and  domain 
dependent  model  types  and  therefore  rates  the  retrieval  performance  with  respect  to  cross-domain  retrieval  tasks. 

18)  Sketch-based  3D  Model  Retrieval 

The  research  team  built  the  largest  sketch-3D  model  benchmark  dataset.  The  benchmark  contains  13,680  sketches  and  8,987 
3D  models,  divided  into  171  distinct  classes.  The  developed  benchmark  is  the  most  extensive  to  date  in  terms  of  the  number  of 
semantic  query  categories  covered  as  well  as  the  variations  of  model  types.  The  benchmark  supports  both  sketch  and  3D 
model  queries,  thus  providing  a  unified  platform  to  test  diverse  3D  model  retrieval  algorithms  belonging  to  either  Query-by- 
Model  or  Query-by-Sketch  3D  retrieval  techniques.  This  benchmark  also  provides  an  important  resource  for  the  community  of 
sketch-based  3D  retrieval  and  will  foster  the  development  of  practical  sketch  based  3D  retrieval  applications. 

Based  on  this  new  developed  benchmark,  the  research  team  organized  a  sketch-based  3D  model  retrieval  contest  in  2014 
Eurographics.  The  task  of  the  contest  is  to  evaluate  the  performance  of  different  sketch-based  3D  model  retrieval  algorithms 


using  a  large  scale  hand-drawn  sketch  query  dataset  on  a  comprehensive  3D  model  dataset.  12  runs  of  6  methods  have  been 
submitted  by  4  groups.  The  research  team  performs  a  comprehensive  comparison  study  and  evaluates  current  state-of-the-art 
sketch-based  retrieval  approaches,  especially  in  terms  of  scalability  using  7  popular  performance  metrics.  This  benchmark  and 
comprehensive  study  will  provide  important  guidance  on  future  research  directions  of  this  research  area. 

19)  3D  Symmetry  Detection 

The  research  team  proposed  a  novel  and  efficient  view-based  symmetry  detection  algorithm.  The  proposed  algorithm  can  find 
symmetry  plane(s)  by  matching  the  viewpoint  entropy  features  of  a  set  of  sample  views  of  a  3D  model  aligned  beforehand  using 
Continuous  Principal  Component  Analysis  (CPCA).  The  experimental  results  show  the  proposed  symmetry  detection  algorithm 
is  more  accurate  (in  terms  of  both  the  positions  of  detected  symmetry  planes  and  sensitivity  to  minor  symmetry  differences), 
efficient,  robust  (e.g.  to  the  number  of  vertices  and  parameter  settings  such  as  view  sampling),  and  versatile  in  finding 
symmetry  planes  of  diverse  models. 

During  the  past  year  (2015),  the  following  scientific  accomplishments  have  been  made: 

20)  Feature  Descriptors  for  Large-scale  Image  Retrieval 

The  research  team  propose  a  novel  cascaded  scalar  quantization  scheme  in  dual  resolution.  They  formulate  the  visual  feature 
matching  as  a  range-based  neighbor  search  problem  and  approach  it  by  identifying  hyper-cubes  with  a  dual-resolution  scalar 
quantization  strategy.  Specifically,  for  each  dimension  of  the  dimension-reduced  feature,  scalar  quantization  is  performed  at 
both  coarse  and  fine  resolutions.  The  scalar  quantization  results  at  the  coarse  resolution  are  cascaded  over  multiple  dimensions 
to  index  an  image  database.  The  scalar  quantization  results  over  multiple  dimensions  at  the  fine  resolution  are  concatenated 
into  a  binary  super-vector  and  stored  into  the  index  list  for  efficient  verification.  The  proposed  cascaded  scalar  quantization 
(CSQ)  method  is  free  of  the  costly  visual  codebook  training  and  thus  is  independent  of  any  image  descriptor  training  set.  The 
index  structure  of  the  CSQ  is  flexible  enough  to  accommodate  new  image  features  and  scalable  to  index  large-scale  image 
database. 

The  research  teams  also  extend  the  concept  of  image  retargeting  and  propose  a  new  image  resizing  approach  that  is  devoted 
to  preserving  the  robust  local  features  in  the  query  image  while  resizing  it.  Based  on  the  extended  concept,  a  novel  mobile- 
visual-search  scheme  is  introduced  to  conduct  the  proposed  approach  to  reduce  the  size  of  the  query  image  for  achieving  low 
bit-rate  visual  search.  Extensive  experiments  show  that  the  proposed  approach  obtains  superior  retrieval  performance  than 
state-of-the-art  image  resizing  approaches  at  the  similar  query  size;  meanwhile,  it  is  cost  effective  in  terms  of  processing  time. 

21)  Image  Representation  for  Large-scale  Image  Retrieval 

The  research  team  propose  a  hierarchical  method  to  construct  VLAD  descriptor  (HVLAD)  to  inherit  the  benefit  of  finer  division 
bringing  by  larger  vocabulary  while  preserving  the  same  dimension  with  the  original  VLAD  descriptor.  In  the  proposed  HVLAD 
descriptor,  by  generating  sub-words  to  each  words  of  the  coarse  vocabulary  that  is  adopted  to  build  original  VLAD  descriptor,  a 
hidden  layer  visual  vocabulary  is  constructed.  With  the  hidden  layer  vocabulary,  the  feature  space  is  finely  divided.  The  residual 
vectors  between  local  features  and  sub-words  are  first  aggregated  at  the  hidden  layer  and  then  are  aggregated  to  the  coarse 
layer.  In  addition,  the  research  team  proposes  a  number  of  residual  codebooks  descended  from  the  original  clusters.  Then  local 
descriptors  assigned  to  the  same  cluster  are  distinguished  by  their  residuals  and  divided  into  finer  clusters.  Through  these 
codebooks,  the  difference  vector  between  primary  residual  and  its  closest  visual  word  in  the  residual  codebooks  is  calculated 
and  denoted  as  the  secondary  residual.  By  pooling  them  with  the  primary  ones,  the  fine  residuals  with  more  discriminative 
information  are  obtained.  Furthermore,  the  fine  residuals  are  aggregated  into  one  vector  through  the  two-step  aggregation, 
keeping  the  same  dimension  as  the  original. 

22)  Index  Strategy  for  Image  Retrieval 

The  research  team  propose  cross  indexing  with  grouplets,  where  the  core  idea  is  to  view  the  database  images  as  a  set  of 
grouplets,  each  of  which  is  defined  as  a  group  of  highly  relevant  images.  Because  a  grouplet  groups  similar  images  together, 
the  number  of  grouplets  is  smaller  than  the  number  of  images,  thus  naturally  leading  to  less  memory  cost.  Moreover,  the 
definition  of  a  grouplet  could  be  based  on  customized  relations,  allowing  for  seamless  integration  of  advanced  image  features 
and  data  mining  techniques  like  the  deep  convolutional  neural  network  (DCNN)  in  off-line  indexing.  To  validate  the  proposed 
framework,  the  research  team  constructs  three  different  types  of  grouplets,  which  are  respectively  based  on  local  similarity, 
regional  relation,  and  global  semantic  modeling.  Extensive  experiments  on  public  benchmark  datasets  demonstrate  the 
efficiency  and  superior  performance  of  the  proposed  approach.  Moreover,  the  research  team  proposes  a  fast  image  retrieval 
framework  to  speed  up  the  online  retrieval  process.  To  this  end,  an  impact  score  for  local  features  is  proposed  in  the  first  place, 
which  considers  multiple  properties  of  a  local  feature,  including  TF-IDF,  scale,  saliency,  and  ambiguity.  Then,  to  decrease 
memory  consumption,  the  impact  score  is  quantized  to  an  integer,  which  leads  to  a  novel  inverted  index  organization,  called  Q- 
Index.  Importantly,  based  on  the  impact  score,  two  closely  complementary  strategies  are  introduced:  query  pruning  and  early 
termination.  On  one  hand,  query  pruning  discards  less  important  features  in  the  query.  On  the  other  hand,  early  termination 
visits  indexed  features  only  with  high  impact  scores,  resulting  in  the  partial  traversing  of  the  inverted  index.  The  proposed 
approach  is  tested  on  two  benchmark  datasets  populated  with  an  additional  1  million  images  to  account  as  negative  examples. 
Compared  with  full  traversal  of  the  inverted  index,  it  shows  that  the  developed  system  is  capable  of  visiting  less  than  1 0%  of  the 
“should-visit”  postings,  thus  achieving  a  significant  speed-up  in  query  time  while  providing  competitive  retrieval  accuracy. 

23)  Information  Fusion  for  Image  Retrieval 

For  vocabulary  tree  based  image  retrieval,  the  research  team  propose  a  semantic-aware  co-indexing  algorithm  to  jointly  embed 
two  strong  cues  into  the  inverted  indexes:  1 )  local  invariant  features  that  are  robust  to  delineate  low-level  image  contents,  and 
2)  semantic  attributes  from  large-scale  object  recognition  that  may  reveal  image  semantic  meanings.  For  an  initial  set  of 
inverted  indexes  of  local  features,  the  research  team  utilizes  1000  semantic  attributes  to  filter  out  isolated  images  and  insert 
semantically  similar  images  to  the  initial  set.  Encoding  these  two  distinct  cues  together  effectively  enhances  the  discriminative 


capability  of  inverted  indexes.  Such  co-indexing  operations  are  totally  off-line  and  introduce  small  computation  overhead  to 
online  query  cause  only  local  features  but  no  semantic  attributes  are  used  for  query.  Experiments  and  comparisons  with  recent 
retrieval  methods  on  3  datasets,  i.e.,  UKbench,  Holidays,  OxfordSK,  and  1.3  million  images  from  Flickr  as  distractors,  manifest 
the  competitive  performance  of  the  proposed  method.  In  addition,  the  research  team  propose  a  score-level  fusion  scheme 
based  on  a  simple  motivation:  the  score  curve  of  a  good  feature  is  “L”  shaped,  while  that  of  a  bad  feature  is  gradually  dropping. 
In  a  nut-shell,  the  score  curves  are  firstly  normalized  by  reference  curves  trained  on  irrelevant  data,  which  are  expected  to 
approximate  the  tails  of  the  initial  score  curves.  Then,  feature  effectiveness  is  estimated  as  negatively  related  to  the  area  under 
the  normalized  score  curve.  In  the  proposed  method,  the  offline  operation  is  independent  on  the  test  database,  making  it  well 
suited  to  dynamic  systems.  More  importantly,  the  proposed  method  identifies  “good”  and  “bad”  features  on-the-fly,  and  the 
results  are  competitive  to  the  state-of-the-arts  on  three  datasets. 

24)  Post  Processing  for  Web  Image  Search 

The  research  team  propose  a  new  attribute-assisted  reranking  method  based  on  hypergraph  learning.  They  first  train  several 
classifiers  for  all  the  pre-defined  attributes  and  each  image  is  represented  by  attribute  feature  consisting  of  the  responses  from 
these  classifiers.  Different  from  the  existing  methods,  a  hypergraph  is  then  used  to  model  the  relationship  between  images  by 
integrating  low-level  features  and  attribute  features. 

The  research  team  also  improve  the  hypergraph  learning  method  approach  presented  in  by  adding  a  regularizer  on  the 
hyperedge  weights  which  performs  an  implicit  selection  on  the  semantic  attributes.  This  makes  the  proposed  approach  much 
more  robust  and  discriminative  for  image  representation  as  noisy  attributes  will  be  removed  and  informative  ones  will  be 
selected.  Comprehensive  experiments  have  been  conducted  to  empirically  analyze  the  proposed  method  on  more  than  1,000 
queries  and  1  million  images.  The  experimental  results  validate  the  effectiveness  of  the  proposed  method. 

25)  Action  Recognition  for  Video  Application 

The  research  team  propose  a  novel  fine-grained  action  recognition  pipeline  by  interaction  part  proposal  and  discriminative  mid¬ 
level  part  mining.  Firstly,  a  large  number  of  candidate  object  regions  are  generated  by  using  off-the-shelf  object  proposal  tool,  e. 
g.,  BING.  Secondly,  these  object  regions  are  matched  and  tracked  across  frames  to  form  a  large  spatio-temporal  graph  based 
on  the  appearance  matching  and  the  dense  motion  trajectories  through  them.  The  research  team  then  proposes  an  efficient 
approximate  graph  segmentation  algorithm  to  partition  and  filter  the  graph  into  consistent  local  dense  sub-graphs.  These  sub¬ 
graphs,  which  are  spatiotemporal  sub-volumes,  represent  the  candidate  interaction  parts.  Finally,  the  research  team  mines 
discriminative  mid-level  part  detectors  from  the  features  computed  over  the  candidate  interaction  parts.  Extensive  experiments 
have  been  conducted  on  human-object  interaction  datasets.  The  experimental  results  demonstrate  that  the  proposed 
framework  achieves  consistent  improvements  over  the  state-of-the-art  action  recognition  accuracies  on  the  benchmarks, 
without  using  any  object  annotation.  In  addition,  the  research  team  proposes  to  use  descriptors  at  Higher  Semantic  levels  in 
combination  with  the  low-level  dynamic  ones  for  action  recognition. 

26)  Feature  Quantization 

The  research  team  propose  to  construct  a  supporting  visual  word  table  for  all  visual  words  by  visual  word  expansion.  Given  the 
initial  quantization  result,  multiple  approximate  nearest  visual  words  are  identified  by  checking  supporting  visual  word  table, 
which  benefits  the  retrieval  recall.  Moreover,  a  matching  verification  scheme  based  on  binary  SIFT  (BSIFT)  is  presented.  The 
L2 -distance  between  original  SIFT  descriptors  is  demonstrated  to  be  well  kept  with  the  metric  of  Hamming  distance  between  the 
corresponding  binary  SIFT  vectors.  With  the  BSIFT  verification,  false  positive  matches  can  be  effectively  and  efficiently 
identified  and  removed,  which  greatly  improves  the  accuracy  of  large-scale  image  search.  The  proposed  approach  is  evaluated 
by  conducting  partial-duplicate  image  search  on  a  one-million  image  database.  The  experimental  results  demonstrate  the 
effectiveness  and  efficiency  of  the  proposed  scheme. 

27)  Image  Search  Results  Quality  Assessment 

The  research  team  quantitatively  study  and  formulate  the  image  search  result  preference  learning  problem.  A  novel  framework 
and  a  set  of  valuable  features  to  automatically  compare  the  quality  of  image  search  result  lists  are  proposed.  A  general 
preference  learning  model  and  a  query  dependent  preference  learning  model  are  proposed.  The  proposed  approach  has  been 
tested  on  a  variety  of  applications  including  optimal  search  engine  selection,  merging  of  search  result  lists,  selecting  the  best 
visual  feature  and  reranking  approach  for  each  individual  query,  and  synonymous  query  suggestion.  Extensive  experimental 
results  have  demonstrated  the  effectiveness  of  the  proposed  approach  and  its  promising  applications  on  reranking  feature  and 
model  selection,  merging  of  image  search  results,  as  well  as  query  suggestion.  This  work  will  explicitly  guide  the  research  in 
visual  reranking  ability  estimation  and  provide  a  path  for  query  difficulty  modeling. 

28)  Image  Search  Reranking 

The  research  team  introduce  a  learning-based  reranking  method  “Topic  Aware  Reranking”  (TARerank)  to  refine  text-based 
image  search  results.  This  method  not  only  takes  topic  importance  into  consideration,  but  also  directly  learns  a  reranking  model 
by  optimizing  a  criterion  related  to  reranking  performance  in  terms  of  both  relevance  and  diversity  in  one  stage  simultaneously. 
To  better  model  the  hierarchical  topic  structure  of  search  results  and  describe  the  relevance  and  diversity  in  one  criterion 
seamlessly,  NCTC  is  also  proposed  to  quantify  the  hierarchical  TC.  Compared  with  the  two-step  optimization  in  other  diversified 
reranking  methods,  TARerank  can  achieve  the  joint  optimum  of  improving  relevance  and  diversity.  Besides,  the  learning 
procedure  can  bridge  the  gap  between  low-level  visual  feature  diversity  and  high-level  semantic  topic  diversity  to  some  extent. 
These  two  advantages  ensure  the  superiority  of  TARerank.  By  conducting  extensive  experiments  on  a  Web  image  dataset,  the 
research  team  has  demonstrated  the  effectiveness  of  the  proposed  method.  This  method  will  be  a  promising  new  paradigm  for 
Web  image  search  reranking. 

29)  Sketch-based  3D  Model  Retrieval 

In  this  work,  the  research  team  propose  and  implement  a  novel  3D  sketching  virtual  drawing  “board”  (software),  which  allows 


users  to  freely  draw  3D  sketches  in  the  air  (a  real  3D  space).  Based  on  this  developed  3D  sketching  virtual  drawing  board,  the 
first  human  3D  sketch  dataset  is  collected.  The  research  team  also  introduce  a  3D  sketch-based  3D  model  retrieval  system  to 
solve  the  matching  problem  between  3D  sketches  and  models.  And  the  proposed  approach  shows  promising  application 
potential  for  3D  sketch  understanding  or  recognition,  large  scale  3D  model  search,  and  on-line  3D  model  shopping,  etc. 

This  work  is  the  first  attempt  to  explore  3D  sketching  in  a  3D  space  and  to  develop  an  innovative  retrieval  system  that  enables 
users  to  search  3D  models  based  on  hand-drawn  3D  sketches.  The  implications  of  this  work  could  be  tremendous,  as  3D 
sketching  allows  for  more  direct  communication  in  a  user’s  drawing,  which  could  not  only  enhance  3D  model  retrieval  accuracy, 
but  also  provide  a  possibility  for  other  human  sketch  related  applications,  such  as  virtual  try-on  systems  for  clothes,  glasses  and 
watches.  This  work  will  also  explicitly  guide  the  research  in  3D  sketching  and  provide  a  path  for  large  scale  sketch-based 
image,  video,  and  object  retrieval. 

30)  3D  Symmetry  Detection 

The  research  team  propose  a  novel  and  efficient  view-based  symmetry  detection  algorithm.  The  proposed  algorithm  can  find 
symmetry  plane(s)  by  matching  the  viewpoint  entropy  features  of  a  set  of  sample  views  of  a  3D  model  aligned  beforehand  using 
Continuous  Principal  Component  Analysis  (CPCA).  The  experimental  results  show  the  proposed  symmetry  detection  algorithm 
is  more  accurate  (in  terms  of  both  the  positions  of  detected  symmetry  planes  and  sensitivity  to  minor  symmetry  differences), 
efficient,  robust  (e.g.  to  the  number  of  vertices  and  parameter  settings  such  as  view  sampling),  and  versatile  in  finding 
symmetry  planes  of  diverse  models. 
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etc.)  sheds  bright  light  on  many  promising  applications  in  forensics,  surveillance,  3D  animation, 
mobile  visual  search,  and  3D  model/object  search.  Compared  with  the  general  images,  partial- 
duplicate  images  have  some  intrinsic  properties  such  as  high  repeatability  of  local  features, 
consistent  local  patch  appearance,  and  stable  spatial  configuration.  Compared  with  the  general 


2D  objects,  3D  models/objects  consist  of  3D  data  information  (typically  a  list  of  vertices  and 
faces)  to  represent  3D  objects.  However,  these  unique  properties  of  partial-duplicate  images  and 
3D  models  have  not  been  well  exploited  to  design  effective  and  efficient  search  algorithms. 
Because  of  this,  existing  works  for  large-scale  partial-duplicate  image  retrieval  and  3D  model 
retrieval  suffer  from  two  major  problems:  low  accuracy  and  low  efficiency.  These  problems 
make  them  fall  far  below  many  applications’  requirement.  This  project  has  investigated  many 
key  problems  in  large-scale  partial-duplicate/similar  image  and  3D  model  retrieval:  feature 
descriptor  problem,  image  representation  problem,  index  strategy  problem,  feature  quantization 
problem,  image  search  results  quality  assessment  problem,  image  search  reranking  problem, 
sketch-based  3D  model  retrieval  problem,  and  related  search  problems  and  has  proposed  a  series 
of  effective  and  efficient  approaches  to  solve  them. 

3.  Summary  of  the  scientific  accomplishments 

The  key  elements  developed  in  this  project  will  make  significant  contributions  to  improve  the 
performance  of  large  scale  partial  duplicate  image  retrieval  system,  large  scale  3D  model 
retrieval  system,  and  personalized  image  retrieval.  The  research  teams  are  one  of  the  first  (to 
their  best  knowledge)  in  the  field  who 

•  propose  a  novel  strategy  to  generate  a  reliable,  easily  updated  visual  dictionary  with  low 
computational  cost.  In  our  method,  no  visual  codebook  is  needed  to  be  trained  and  the 
quantizer  is  independent  of  collections  of  images; 

•  propose  to  formulate  the  codebook  construction  and  contextual  subspace  learning  into  one 
optimization  problem  and  embed  semantic  information; 

•  apply  visual  word  expansion  approach  to  reduce  the  feature  quantization  error  and  boost  the 
retrieval  recall; 

•  propose  to  embed  the  spatial  information  between  local  features  into  the  inverted  index  file 
efficiently; 

•  propose  to  build  geometric  square  coding  and  geometric  fan  coding  together  to  fully  capture 
the  global  geometric  context  of  local  features  in  an  image  and  effectively  discover  false 
feature  matches  between  images. 

•  propose  a  novel  flexible  scale  invariant  feature  transform  (SIFT)  binarization  (FSB) 
algorithm; 

•  propose  to  explore  the  use  of  partial  binary  descriptors  as  direct  codebook  indices  (addresses); 

•  introduce  a  novel  IDF  family  by  the  use  of  Lp-norm  pooling  technique; 

•  propose  a  novel  attribute-assisted  retrieval  model  for  reranking  images; 

•  build  the  largest  comprehensive  3D  shape  database  that  contains  different  types  of  models 
including  generic,  articulated,  CAD  and  architecture  models; 

•  construct  the  largest  sketch-3D  model  benchmark  that  contains  a  large  number  of  diverse 
types  of  sketches  and  models; 

•  apply,  compare,  and  analyze  state-of-the-art  3D  model  retrieval  approaches  on  the  collected 
sketch  and  3D  model  benchmark; 


•  propose  a  simple  and  efficient  view-based  3D  symmetry  detection  method. 

•  propose  a  novel  matching  verification  scheme  based  on  binary  SIFT  (BSIFT); 

•  propose  two  novel  binary  feature  descriptors:  COGE  and  Edge-SIFT  and  investigate 
representing  the  spatial  context  of  local  features  into  binary  codes; 

•  encode  local  invariant  features  and  high  level  semantic  attributes  together  to  effectively 
enhance  the  discriminative  capability  of  inverted  indexes; 

•  use  the  semantic  classification  information  to  reduce  the  semantic  gap,  as  well  as  to 
adequately  utilize  the  better-performing  global  feature  matching  to  improve  sketch-based  3D 
model  retrieval  efficiency; 

•  propose  a  3D  visual  complexity  metric  based  on  the  viewpoint  entropy  distribution  of  a  set  of 
sample  views; 

•  explore  the  correlations  between  users’  different  touch  behaviors  and  human  attention, 
investigate  their  contribution  to  the  human  eye  fixation,  and  propose  a  novel  supervised 
learning  approach  to  learn  good  image  saliency  maps  from  multiple  touch  behaviors. 

During  the  past  year  (2012),  the  following  scientific  accomplishments  have  been  made: 

1)  Scalar  Quantization 

The  research  team  investigate  bit  vector  quantization  algorithms  to  address  the  “high 
computational  cost”,  “limited  reliability”,  and  “update  inefficiency”  problems  in  the  codebook 
generation  stage.  A  novel  feature  quantization  algorithm,  scalar  quantization,  is  proposed.  With 
scalar  quantization,  a  local  feature  is  quantized  to  a  descriptive  and  discriminative  bit- vector,  of 
which  the  first  tens  of  bits  are  taken  out  as  code  word.  Our  quantizer  is  independent  of 
collections  of  images.  In  addition,  the  result  of  scalar  quantization  naturally  lends  itself  to  adapt 
to  the  classic  inverted  file  structure  for  image  indexing.  Moreover,  the  quantization  error  can  be 
flexibly  reduced  and  controlled  by  efficiently  enumerating  nearest  neighbors  of  code  words. 

2)  Discriminative  Codebook  Learning 

The  research  team  propose  a  novel  supervised  discriminative  codebook  learning  method,  which 
can  find  a  contextual  subspace  to  embed  the  semantic  information  into  codebook  generation  and 
learn  a  contextual  subspace  and  discriminative  codebook  simultaneously.  In  the  learned  new 
space,  images  from  different  classes  can  be  well  separated  and  images  from  the  same  class  are 
close  to  each  other.  The  research  team  applied  the  proposed  method  on  Web  image  search  re¬ 
ranking  problem  and  the  experimental  results  on  two  real  Web  image  search  datasets  have 
demonstrated  the  effectiveness  of  our  approach  and  its  superiority  than  other  state-of-the-art 
codebook  learning  methods. 

3)  Visual  Word  Expansion  and  Binary  SIFT 

The  research  team  propose  a  novel  visual  word  expansion  approach  to  improve  the  feature 
quantization  accuracy  and  boost  the  retrieval  recall.  The  proposed  visual  word  expansion  scheme 
is  based  on  the  observation  that  the  expected  nearest  visual  word  to  a  test  feature  is  always  close 
to  the  approximate  nearest  visual  word,  which  can  be  efficiently  identified  by  the  hierarchical  k- 


NN  search.  Experiments  on  image  search  in  million-scale  dataset  demonstrate  the  effectiveness 
of  the  proposed  visual  word  expansion  approach. 

Moreover,  the  research  team  design  a  new  scheme  to  transform  a  SIFT  descriptor  to  a  binary  bit 
stream,  called  binary  SIFT.  Extensive  study  with  large-scale  (trillion)  samples  reveal  that  the 
generated  binary  SIFT  effectively  keeps  the  distance  metric  of  the  original  SIFT  descriptor. 

4)  Embedding  Spatial  Context  into  Inverted  File 

The  research  team  explore  two  novel  ways  to  embed  the  spatial  information  between  local 
features  into  the  inverted  index  file.  A  one-one  relationship  approach  is  proposed  to  record 
feature  tuples  into  the  inverted  file,  which  consist  of  a  pair  of  single  features  and  their 
corresponding  spatial  relationship.  A  one-multiple  relationship  approach  is  proposed  to  record 
the  spatial  relationship  between  a  single  feature  and  its  surrounding  features,  which  are  clustered 
into  different  groups  based  on  their  locations.  With  the  embedded  spatial  information  in  the 
inverted  index  file,  feature  matching  with  geometric  verification  can  be  performed  implicitly, 
efficiently,  and  effectively  in  the  inverted  file  traversing  step,  which  make  the  proposed  methods 
more  suitable  for  large-scale  image  search. 

5)  Spatial  and  Geometric  Coding  for  Geometric  Verification 

The  research  team  investigate  two  different  coding  algorithms  “Spatial  Coding”  and  “Geometric 
Coding”  to  achieve  fast  and  accurate  geometric  verification  of  local  features  in  the  post 
processing  step.  The  proposed  spatial  coding  algorithm  encodes  the  relative  positions  between 
each  pair  of  features  along  the  horizontal  (A-axis)  and  vertical  (T-axis)  direction  by  generating 
two  binary  spatial  maps  A-map  and  T-map.  The  proposed  geometric  coding  method  consists  of 
geometric  square  coding  and  geometric  fan  coding,  which  describe  the  spatial  relationships  of 
local  features  (e.g.  inside  or  outside  of  geometric  square  and  fan)  into  //-map,  V-map,  and  5-map 
for  global  verification  to  remove  geometrically  inconsistent  feature  matches.  These  two 
approaches  are  not  only  computationally  efficient,  but  also  effective  in  detecting  partial-duplicate 
images  with  rotation,  scale  changes,  partial-occlusion,  and  background  clutter. 

6)  Applications 

During  the  exploration  of  solutions  to  solve  the  above  scientific  barriers  in  the  partial  duplicate 
image  retrieval,  the  research  team  find  some  other  interesting  applications,  which  may  share 
similar  problems,  such  as  license  plate  detection  (characters  in  different  license  plates  are 
duplicates  of  each  other),  similar  song  retrieval,  and  3D  image  search.  Motivated  by  these 
similarities,  several  new  approaches  are  proposed  for  these  applications  by  bringing  the  ideas  in 
the  partial  duplication  image  retrieval. 

Observing  that  characters  in  different  license  plates  are  duplicates  of  each  other,  we  bring  in  the 
idea  of  bag-of-words  (BoW)  model  popularly  into  license  plate  detection.  The  research  team 
propose  a  novel  scheme  to  automatically  locate  license  plate  by  principal  visual  word  discovery 
and  local  feature  matching.  For  large  scale  similar  song  retrieval,  the  research  team  proposed  to 
utilize  beat-aligned  chroma  patches  and  applied  location  coding  scheme  to  encode  the  location 


relationships  among  beat-aligned  chroma  patches  in  a  song.  Our  approach  is  both  efficient  and 
effective  to  discover  true  matches  of  beat  chroma  patches  between  songs  with  low  computational 
cost.  In  the  3D  image  retrieval,  the  research  team  propose  a  sketch-based  3D  model  retrieval 
algorithm  by  utilizing  viewpoint  entropy-based  adaptive  view  clustering  and  shape  context 
matching.  The  algorithm  is  tested  on  a  latest  sketch-based  3D  model  retrieval  benchmark  and  the 
results  demonstrate  the  superior  performances  and  advantages  of  our  algorithm. 

During  the  past  year  (2013),  the  following  scientific  accomplishments  have  been  made: 

7)  Feature  Quantization 

The  research  team  propose  to  construct  a  supporting  visual  word  table  for  all  visual  words  by 
visual  word  expansion.  Given  the  initial  quantization  result,  multiple  approximate  nearest  visual 
words  are  identified  by  checking  supporting  visual  word  table,  which  benefits  the  retrieval  recall. 
Moreover,  a  matching  verification  scheme  based  on  binary  SIFT  (BSIFT)  is  presented.  The  Li- 
distance  between  original  SIFT  descriptors  is  demonstrated  to  be  well  kept  with  the  metric  of 
Hamming  distance  betThe  research  teamen  the  corresponding  binary  SIFT  vectors.  With  the 
BSIFT  verification,  false  positive  matches  can  be  effectively  and  efficiently  identified  and 
removed,  which  greatly  improves  the  accuracy  of  large-scale  image  search.  The  proposed 
approach  is  evaluated  by  conducting  partial-duplicate  image  search  on  a  one-million  image 
database.  The  experimental  results  demonstrate  the  effectiveness  and  efficiency  of  the  proposed 
scheme. 

8)  Binary  Feature  Descriptors  for  Image  Retrieval 

The  research  team  propose  two  novel  binary  feature  descriptors:  COGE  and  Edge-SIFT.  The 
proposed  COGE  exploits  the  anisotropy  and  the  non-uniformity  of  the  underlying  gradient 
distributions.  Therefore,  it  exhibits  better  results  than  state-of-the-art  methods  with  respect  to 
distinctiveness,  robustness,  and  storage  cost.  The  proposed  Edge-SIET  is  generated  from  the 
binary  edge  maps  of  scale  and  orientation-normalized  image  patches.  By  preserving  both 
locations  and  orientations  of  edges  and  compressing  the  sparse  binary  edge  maps  with  a  boosting 
strategy,  Edge-SIET  shows  strong  discriminative  power  with  compact  representation. 

In  addition,  the  research  team  investigate  representing  the  spatial  context  of  local  features  into 
binary  codes  to  implicitly  achieve  geometric  verification  by  efficient  comparison  of  the  binary 
codes.  Moreover,  a  weighted  Hamming  distance  ranking  algorithm  (WhRank)  to  rank  the  binary 
codes  of  hashing  methods  is  developed.  By  assigning  different  bit-level  weights  to  different  hash 
bits,  the  returned  binary  codes  are  ranked  at  a  finer-grained  binary  code  level.  Experiments  on 
several  state-of-the-art  benchmark  data  sets  with  million-scale  distractor  images  demonstrate  the 
effectiveness  of  the  proposed  algorithms. 

9)  Semantic-aware  Co-indexing  for  Image  Search 

The  research  team  propose  a  novel  approach  for  embedding  semantic  attributes  into  the  pipeline 
of  image  search.  Inverted  indexes  in  image  retrieval  not  only  allow  fast  access  to  database 
images  but  also  summarize  all  knowledge  about  the  database,  so  that  their  discriminative 


capacity  largely  determines  the  retrieval  performance.  For  vocabulary  tree  based  image  retrieval, 
a  semantic-aware  co-indexing  algorithm  is  proposed  to  jointly  embed  two  strong  cues  into  the 
inverted  indexes:  1)  local  invariant  features  that  are  robust  to  delineate  low-level  image  contents, 
and  2)  semantic  attributes  from  large-scale  object  recognition  that  may  reveal  image  semantic 
meanings.  Encoding  these  two  distinct  cues  together  effectively  enhances  the  discriminative 
capability  of  inverted  indexes.  Such  co-indexing  operations  are  totally  off-line,  and  therefore 
introduce  small  computation  overhead  to  online  query.  Experiments  and  comparisons  with  recent 
retrieval  methods  have  evidently  demonstrated  the  significant  improvement  in  retrieval 
performance  and  efficiency. 

10)  Sketch-based  Image/3D  Model  Retrieval 

The  research  team  propose  a  new  sketch  feature  to  capture  both  the  local  and  global  information 
of  a  sketch.  The  proposed  sketch  feature  can  handle  the  rotation  of  sketches  better  by  further 
integrating  a  set  of  rotation-invariant  global  features  for  a  sketch.  In  addition,  an  intelligent 
sketch  recognizer  through  supervised  learning  is  developed  to  correctly  capture  the  semantic 
meanings  of  users’  sketches.  The  research  team  also  conducts  a  comprehensive  study  of  the 
semantic  gap  between  user  sketches  and  3D  models  and  proposes  a  novel  semantic  sketch-based 
3D  model  search  algorithm  to  bridge  such  a  semantic  gap.  The  experimental  results  confirmed 
that  the  proposed  sketch  recognizer  is  more  robust  to  sketch  rotation  and  can  describe  user 
sketches  well.  In  addition,  the  proposed  semantic  retrieval  approach  significantly  improves  the 
retrieval  accuracy  and  reduces  the  search  time  over  other  state-of-the-art  sketch-based  3D  model 
retrieval  algorithms.  It  further  validated  that  the  proposed  approach  could  bridge  the  semantic 
gap  between  the  diverse  query  sketches  and  3D  models  effectively. 

1 1)  Large  Scale  3D  Model  Retrieval 

The  research  team  propose  a  new  3D  model  retrieval  algorithm  by  adaptive  view  clustering.  A 
novel  3D  visual  complexity  metric  is  formulated  based  on  the  viewpoint  entropy  distribution  of  a 
set  of  uniformly  sampled  views  of  the  3D  model.  Then,  it  guides  adaptive  view  clustering  of  a 
3D  model  to  shortlist  a  set  of  representative  sample  views  for  2D-3D  comparison,  which  largely 
reduces  the  number  of  sample  views  for  comparison  and  therefore  greatly  improves  the 
computational  efficiency.  After  that,  a  shape  context  matching  algorithm  is  used  for  the  2D-3D 
matching  between  the  query  and  the  representative  views  for  each  model.  Experimental  results 
on  several  latest  benchmarks  have  evidently  demonstrated  our  significant  improvement  in 
retrieval  performance  and  efficiency. 

In  addition,  the  research  team  organize  a  “Large  Scale  Sketch-based  3D  Model  Retrieval” 
competition  in  the  Shape  Retrieval  Contest  (SHREC)  2013,  which  is  held  at  the  3D  Object 
Retrieval  (3DOR)  workshop  of  Eurographics  2013.  In  the  meantime,  the  research  team  also 
attend  two  SHREC  contests  on  “Large-Scale  3D  Partial  Shape  Retrieval  Track  Using  Simulated 
Range  Images”  and  “Retrieval  of  3D  Objects  Captured  with  Low-Cost  Depth-Sensing  Cameras”. 
In  these  competitions,  the  proposed  3D  model  retrieval  algorithm  receives  First  Place  in  the 


large-scale  sketch-based  3D  retrieval  track,  First  Place  in  the  range  scan  track,  and  obtains 
Second  Place  in  the  low-cost  depth-sensing  camera  track. 

12)  Touch-based  Personalized  Image  Search 

The  research  team  quantitatively  studied  and  analyzed  human  attention  from  a  variety  of  touch 
behaviors,  and  proposes  a  set  of  valuable  features  from  the  touch  information.  A  series  of 
experiments  are  designed  and  conducted  with  the  conventional  eye-fixation  based  saliency 
serving  as  the  ground  truth.  An  image  browsing  app  is  designed  on  a  touch  mobile  phone  to 
collect  users’  touch  behavior  data.  In  addition,  a  novel  touch  saliency  learning  approach  is 
proposed  to  automatically  learn  the  correlation  between  different  touch  behaviors  and  human  eye 
fixations,  and  then  to  derive  a  good  image  saliency  map  from  a  variety  of  touch  behaviors. 
During  the  process  of  building  a  supervised  learning  model,  the  weights  of  different  human 
touch  behaviors  are  learned,  which  indicate  the  different  contributions  of  these  behaviors  to  the 
user’s  attention  information.  The  experimental  results  have  demonstrated  the  validity  of  the 
study  and  the  potential  and  effectiveness  of  the  proposed  approach. 

13)  Applications 

In  addition,  the  research  team  explored  many  important  related  problems:  1)  “human  movement 
summarization  and  depiction  from  videos”  problem.  Previous  action  summarization  methods  all 
rely  on  3D  motion  capture  or  manually  labeled  data,  without  which  depicting  actions  is  a 
challenging  task.  A  novel  scheme  is  proposed  to  automatically  summarize  and  depict  human 
movements  from  2D  videos  without  3D  motion  capture  or  manually  labeled  data.  The  proposed 
method  can  successfully  segment  videos  into  sub-actions  with  an  effective  streamline  matching 
scheme,  track  points  on  body  parts,  and  depict  the  human  articulated  motion  with  arrows  and 
motion  particles.  2)  “image  matching”  problem.  A  new  matching  method,  Semantic -Spatial 
Matching  (SSM)  is  proposed.  SSM  conducts  region  matching  by  considering  both  the  spatial 
layout  and  the  semantic  content  information.  SSM  has  the  advantage  not  only  being  robust  to 
rotation,  flipping  and  other  variances,  but  also  simple  and  easy  for  implementation.  Experiments 
on  two  benchmark  datasets  demonstrate  its  effectiveness  in  object  and  scene  classifications.  3) 
“3D  scene  reconstruction”  problem.  The  performance  of  the  PMVS  (Patch-based  Multi- View 
Stereo  software)  for  scene  reconstruction  from  stereo  pairs  of  scenes  captured  by  a  simple  3D 
camera  has  been  analyzed.  A  Canny  feature-based  PMVS  algorithm  and  a  preliminary  disparity 
map-based  stereo  reconstruction  algorithm  are  proposed.  Both  proposed  approaches  are 
promising  for  related  applications  which  require  effective  3D  scene  reconstruction  from  a  set  of 
sparsely  sampled  pairs.  4)  “image  annotation”  problem.  An  automatic  image  annotation 
approach  using  semantic  relevance  is  proposed.  It  constructs  an  improved  probabilistic  model  to 
characterize  different  regions’  contributions  to  the  semantics  more  accurately  based  on  the 
spatial,  visual  and  contextual  information  of  the  region.  And  it  also  helps  expand  the  coverage 
of  the  semantic  concept  with  semantic  relevance  information. 

During  the  past  year  (2014),  the  following  scientific  accomplishments  have  been  made: 


14)  Compact  Binary  Feature  Descriptors  for  Image  Retrieval 

The  research  team  propose  a  novel  flexible  scale  invariant  feature  transform  (SIFT)  binarization 
(FSB)  algorithm  for  large-scale  image  search.  The  FSB  algorithm  explores  the  magnitude 
patterns  of  SIFT  descriptor.  It  is  unsupervised  and  the  generated  binary  codes  are  demonstrated 
to  be  dispreserving.  Besides,  a  new  searching  strategy  is  investigated  to  find  target  features  based 
on  the  cross-indexing  in  the  binary  SIFT  space  and  original  SIFT  space.  The  experiments  on 
large-scale  partial  duplicate  image  retrieval  system  demonstrate  the  effectiveness  and  efficiency 
of  the  proposed  algorithm. 

In  addition,  the  research  team  explored  the  use  of  partial  binary  descriptors  as  direct  codebook 
indices  (addresses).  Typically,  binary  descriptors  are  clustered  into  codewords  and  quantized 
with  Hamming  distance,  following  the  conventional  bag-of-words  strategy.  However,  such 
codewords  formulated  in  Hamming  space  do  not  present  obvious  indexing  and  search 
performance  improvement  as  compared  to  the  Euclidean  codewords.  Without  explicit  codeword 
construction,  the  proposed  approach  performs  to  build  multiple  index  tables  which  concurrently 
check  for  collision  of  the  same  hash  values.  The  evaluation  is  performed  on  two  public  image 
datasets  and  the  experimental  results  demonstrate  the  indexing  efficiency  and  retrieval  accuracy 
of  our  approach. 

15)  TF-IDF  strategy  for  Image  Retrieval 

The  research  team  introduced  a  novel  IDF  family  by  the  use  of  Lp-norm  pooling  technique. 
Carefully  designed,  the  proposed  IDF  considers  the  term  frequency,  document  frequency,  the 
complexity  of  images,  as  well  as  the  codebook  information.  The  research  team  further  proposes  a 
parameter  tuning  strategy,  which  helps  to  produce  optimal  balancing  between  TF  and  pIDF 
weights,  yielding  the  so-called  Lp-norm  IDF  (pIDF).  Further,  by  counting  for  the  term-frequency 
in  each  image,  the  proposed  pIDF  helps  to  alleviate  the  visual  word  burstiness  phenomenon. 

Moreover,  the  team  initializes  to  embed  multiple  binary  features  at  indexing  level.  To  model 
correlation  between  features,  a  multi-IDF  scheme  is  introduced,  through  which  different  binary 
features  are  coupled  into  the  inverted  file.  As  an  extension,  the  research  team  also  explores  the 
fusion  of  binary  color  feature  into  image  retrieval.  The  joint  integration  of  the  SIFT  visual  word 
and  binary  features  greatly  enhances  the  precision  of  visual  matching,  reducing  the  impact  of 
false  positive  matches. 

16)  An  attribute-assisted  Reranking  Model  for  Web  Image  Search 

The  research  team  proposed  a  novel  attribute-assisted  retrieval  model  for  reranking  images. 
Based  on  the  classifiers  for  all  the  predefined  attributes,  each  image  is  represented  by  an  attribute 
feature  consisting  of  the  responses  from  these  classifiers.  A  hypergraph  is  used  to  model  the 
relationship  between  images  by  integrating  low-level  visual  features  and  semantic  attribute 
features.  And  hypergraph  ranking  is  applied  to  re-order  the  images.  Its  basic  principle  is  that 
visually  similar  images  should  have  similar  ranking  scores.  A  visual-attribute  joint  hypergraph 
learning  approach  has  been  proposed  to  simultaneously  explore  two  information  sources.  The 
extensive  experiments  have  been  conducted  on  1,000  queries  in  MSRA-MM  V2.0  dataset.  The 


experimental  results  demonstrate  the  effectiveness  of  the  proposed  attribute-assisted  Web  image 
search  reranking  method. 

17)  Large  Scale  Comprehensive  3D  Model  Retrieval 

The  research  team  built  a  Large  Scale  Comprehensive  3D  model  Benchmark  dataset 
(SHREC14LSGTB).  This  3D  shape  dataset  contains  8,987  models  in  a  variety  of  types  (generic, 
articulated,  CAD  and  architecture  models)  and  classified  into  171  categories.  This  work  is  the 
first  work  to  integrate  existing  3D  model  datasets  to  form  a  new,  larger  benchmark  corpus  for 
comprehensive  3D  shape  retrieval.  The  newly  created  benchmark  is  the  most  extensive  to  date  in 
terms  of  the  number  of  semantic  query  categories  covered  as  well  as  the  variations  of  model 
types.  In  particular,  it  combines  generic  and  domain  dependent  model  types  and  therefore  rates 
the  retrieval  performance  with  respect  to  cross-domain  retrieval  tasks. 

18)  Sketch-based  3D  Model  Retrieval 

The  research  team  built  the  largest  sketch-3D  model  benchmark  dataset.  The  benchmark 
contains  13,680  sketches  and  8,987  3D  models,  divided  into  171  distinct  classes.  The  developed 
benchmark  is  the  most  extensive  to  date  in  terms  of  the  number  of  semantic  query  categories 
covered  as  well  as  the  variations  of  model  types.  The  benchmark  supports  both  sketch  and  3D 
model  queries,  thus  providing  a  unified  platform  to  test  diverse  3D  model  retrieval  algorithms 
belonging  to  either  Query-by-Model  or  Query-by-Sketch  3D  retrieval  techniques.  This 
benchmark  also  provides  an  important  resource  for  the  community  of  sketch-based  3D  retrieval 
and  will  foster  the  development  of  practical  sketch  based  3D  retrieval  applications. 

Based  on  this  new  developed  benchmark,  the  research  team  organized  a  sketch-based  3D  model 
retrieval  contest  in  2014  Eurographics.  The  task  of  the  contest  is  to  evaluate  the  performance  of 
different  sketch-based  3D  model  retrieval  algorithms  using  a  large  scale  hand-drawn  sketch 
query  dataset  on  a  comprehensive  3D  model  dataset.  12  runs  of  6  methods  have  been  submitted 
by  4  groups.  The  research  team  performs  a  comprehensive  comparison  study  and  evaluates 
current  state-of-the-art  sketch-based  retrieval  approaches,  especially  in  terms  of  scalability  using 
7  popular  performance  metrics.  This  benchmark  and  comprehensive  study  will  provide  important 
guidance  on  future  research  directions  of  this  research  area. 

19)  3D  Symmetry  Detection 

The  research  team  proposed  a  novel  and  efficient  view-based  symmetry  detection  algorithm.  The 
proposed  algorithm  can  find  symmetry  plane(s)  by  matching  the  viewpoint  entropy  features  of  a 
set  of  sample  views  of  a  3D  model  aligned  beforehand  using  Continuous  Principal  Component 
Analysis  (CPC A).  The  experimental  results  show  the  proposed  symmetry  detection  algorithm  is 
more  accurate  (in  terms  of  both  the  positions  of  detected  symmetry  planes  and  sensitivity  to 
minor  symmetry  differences),  efficient,  robust  (e.g.  to  the  number  of  vertices  and  parameter 
settings  such  as  view  sampling),  and  versatile  in  finding  symmetry  planes  of  diverse  models. 


During  the  past  year  (2015),  the  following  scientific  accomplishments  have  been  made: 


20)  Feature  Descriptors  for  Large-scale  Image  Retrieval 

The  research  team  propose  a  novel  cascaded  scalar  quantization  scheme  in  dual  resolution.  They 
formulate  the  visual  feature  matching  as  a  range-based  neighbor  search  problem  and  approach  it 
by  identifying  hyper-cubes  with  a  dual-resolution  scalar  quantization  strategy.  Specifically,  for 
each  dimension  of  the  dimension-reduced  feature,  scalar  quantization  is  performed  at  both  coarse 
and  fine  resolutions.  The  scalar  quantization  results  at  the  coarse  resolution  are  cascaded  over 
multiple  dimensions  to  index  an  image  database.  The  scalar  quantization  results  over  multiple 
dimensions  at  the  fine  resolution  are  concatenated  into  a  binary  super-vector  and  stored  into  the 
index  list  for  efficient  verification.  The  proposed  cascaded  scalar  quantization  (CSQ)  method  is 
free  of  the  costly  visual  codebook  training  and  thus  is  independent  of  any  image  descriptor 
training  set.  The  index  structure  of  the  CSQ  is  flexible  enough  to  accommodate  new  image 
features  and  scalable  to  index  large-scale  image  database. 

The  research  teams  also  extend  the  concept  of  image  retargeting  and  propose  a  new  image 
resizing  approach  that  is  devoted  to  preserving  the  robust  local  features  in  the  query  image  while 
resizing  it.  Based  on  the  extended  concept,  a  novel  mobile-visual-search  scheme  is  introduced  to 
conduct  the  proposed  approach  to  reduce  the  size  of  the  query  image  for  achieving  low  bit-rate 
visual  search.  Extensive  experiments  show  that  the  proposed  approach  obtains  superior  retrieval 
performance  than  state-of-the-art  image  resizing  approaches  at  the  similar  query  size;  meanwhile, 
it  is  cost  effective  in  terms  of  processing  time. 

21)  Image  Representation  for  Large-scale  Image  Retrieval 

The  research  team  propose  a  hierarchical  method  to  construct  VLAD  descriptor  (HVLAD)  to 
inherit  the  benefit  of  finer  division  bringing  by  larger  vocabulary  while  preserving  the  same 
dimension  with  the  original  VLAD  descriptor.  In  the  proposed  HVLAD  descriptor,  by  generating 
sub-words  to  each  words  of  the  coarse  vocabulary  that  is  adopted  to  build  original  VLAD 
descriptor,  a  hidden  layer  visual  vocabulary  is  constructed.  With  the  hidden  layer  vocabulary,  the 
feature  space  is  finely  divided.  The  residual  vectors  between  local  features  and  sub-words  are 
first  aggregated  at  the  hidden  layer  and  then  are  aggregated  to  the  coarse  layer.  In  addition,  the 
research  team  proposes  a  number  of  residual  codebooks  descended  from  the  original  clusters. 
Then  local  descriptors  assigned  to  the  same  cluster  are  distinguished  by  their  residuals  and 
divided  into  finer  clusters.  Through  these  codebooks,  the  difference  vector  between  primary 
residual  and  its  closest  visual  word  in  the  residual  codebooks  is  calculated  and  denoted  as  the 
secondary  residual.  By  pooling  them  with  the  primary  ones,  the  fine  residuals  with  more 
discriminative  information  are  obtained.  Lurthermore,  the  fine  residuals  are  aggregated  into  one 
vector  through  the  two-step  aggregation,  keeping  the  same  dimension  as  the  original. 

22)  Index  Strategy  for  Image  Retrieval 

The  research  team  propose  cross  indexing  with  grouplets,  where  the  core  idea  is  to  view  the 
database  images  as  a  set  of  grouplets,  each  of  which  is  defined  as  a  group  of  highly  relevant 
images.  Because  a  grouplet  groups  similar  images  together,  the  number  of  grouplets  is  smaller 
than  the  number  of  images,  thus  naturally  leading  to  less  memory  cost.  Moreover,  the  definition 


of  a  grouplet  could  be  based  on  customized  relations,  allowing  for  seamless  integration  of 
advanced  image  features  and  data  mining  techniques  like  the  deep  convolutional  neural  network 
(DCNN)  in  off-line  indexing.  To  validate  the  proposed  framework,  the  research  team  constructs 
three  different  types  of  grouplets,  which  are  respectively  based  on  local  similarity,  regional 
relation,  and  global  semantic  modeling.  Extensive  experiments  on  public  benchmark  datasets 
demonstrate  the  efficiency  and  superior  performance  of  the  proposed  approach.  Moreover,  the 
research  team  proposes  a  fast  image  retrieval  framework  to  speed  up  the  online  retrieval  process. 
To  this  end,  an  impact  score  for  local  features  is  proposed  in  the  first  place,  which  considers 
multiple  properties  of  a  local  feature,  including  TF-IDF,  scale,  saliency,  and  ambiguity.  Then,  to 
decrease  memory  consumption,  the  impact  score  is  quantized  to  an  integer,  which  leads  to  a 
novel  inverted  index  organization,  called  Q-Index.  Importantly,  based  on  the  impact  score,  two 
closely  complementary  strategies  are  introduced:  query  pruning  and  early  termination.  On  one 
hand,  query  pruning  discards  less  important  features  in  the  query.  On  the  other  hand,  early 
termination  visits  indexed  features  only  with  high  impact  scores,  resulting  in  the  partial 
traversing  of  the  inverted  index.  The  proposed  approach  is  tested  on  two  benchmark  datasets 
populated  with  an  additional  1  million  images  to  account  as  negative  examples.  Compared  with 
full  traversal  of  the  inverted  index,  it  shows  that  the  developed  system  is  capable  of  visiting  less 
than  10%  of  the  “should-visit”  postings,  thus  achieving  a  significant  speed-up  in  query  time 
while  providing  competitive  retrieval  accuracy. 

23)  Information  Fusion  for  Image  Retrieval 

For  vocabulary  tree  based  image  retrieval,  the  research  team  propose  a  semantic-aware  co¬ 
indexing  algorithm  to  jointly  embed  two  strong  cues  into  the  inverted  indexes:  1)  local  invariant 
features  that  are  robust  to  delineate  low-level  image  contents,  and  2)  semantic  attributes  from 
large-scale  object  recognition  that  may  reveal  image  semantic  meanings.  For  an  initial  set  of 
inverted  indexes  of  local  features,  the  research  team  utilizes  1000  semantic  attributes  to  filter  out 
isolated  images  and  insert  semantically  similar  images  to  the  initial  set.  Encoding  these  two 
distinct  cues  together  effectively  enhances  the  discriminative  capability  of  inverted  indexes.  Such 
co-indexing  operations  are  totally  off-line  and  introduce  small  computation  overhead  to  online 
query  cause  only  local  features  but  no  semantic  attributes  are  used  for  query.  Experiments  and 
comparisons  with  recent  retrieval  methods  on  3  datasets,  i.e.,  UKbench,  Holidays,  OxfordSK, 
and  1.3  million  images  from  Flickr  as  distractors,  manifest  the  competitive  performance  of  the 
proposed  method.  In  addition,  the  research  team  propose  a  score-level  fusion  scheme  based  on  a 
simple  motivation:  the  score  curve  of  a  good  feature  is  “F”  shaped,  while  that  of  a  bad  feature  is 
gradually  dropping.  In  a  nut-shell,  the  score  curves  are  firstly  normalized  by  reference  curves 
trained  on  irrelevant  data,  which  are  expected  to  approximate  the  tails  of  the  initial  score  curves. 
Then,  feature  effectiveness  is  estimated  as  negatively  related  to  the  area  under  the  normalized 
score  curve.  In  the  proposed  method,  the  offline  operation  is  independent  on  the  test  database, 
making  it  well  suited  to  dynamic  systems.  More  importantly,  the  proposed  method  identifies 
“good”  and  “bad”  features  on-the-fly,  and  the  results  are  competitive  to  the  state-of-the-arts  on 
three  datasets. 


24)  Post  Processing  for  Web  Image  Search 

The  research  team  propose  a  new  attribute-assisted  reranking  method  based  on  hypergraph 
learning.  They  first  train  several  classifiers  for  all  the  pre-defined  attributes  and  each  image  is 
represented  by  attribute  feature  consisting  of  the  responses  from  these  classifiers.  Different  from 
the  existing  methods,  a  hypergraph  is  then  used  to  model  the  relationship  between  images  by 
integrating  low-level  features  and  attribute  features. 

The  research  team  also  improve  the  hypergraph  learning  method  approach  presented  in  by 
adding  a  regularizer  on  the  hyperedge  weights  which  performs  an  implicit  selection  on  the 
semantic  attributes.  This  makes  the  proposed  approach  much  more  robust  and  discriminative  for 
image  representation  as  noisy  attributes  will  be  removed  and  informative  ones  will  be  selected. 
Comprehensive  experiments  have  been  conducted  to  empirically  analyze  the  proposed  method 
on  more  than  1,000  queries  and  1  million  images.  The  experimental  results  validate  the 
effectiveness  of  the  proposed  method. 

25)  Action  Recognition  for  Video  Application 

The  research  team  propose  a  novel  fine-grained  action  recognition  pipeline  by  interaction  part 
proposal  and  discriminative  mid-level  part  mining.  Firstly,  a  large  number  of  candidate  object 
regions  are  generated  by  using  off-the-shelf  object  proposal  tool,  e.g.,  BING.  Secondly,  these 
object  regions  are  matched  and  tracked  across  frames  to  form  a  large  spatio-temporal  graph 
based  on  the  appearance  matching  and  the  dense  motion  trajectories  through  them.  The  research 
team  then  proposes  an  efficient  approximate  graph  segmentation  algorithm  to  partition  and  filter 
the  graph  into  consistent  local  dense  sub-graphs.  These  sub-graphs,  which  are  spatiotemporal 
sub-volumes,  represent  the  candidate  interaction  parts.  Finally,  the  research  team  mines 
discriminative  mid-level  part  detectors  from  the  features  computed  over  the  candidate  interaction 
parts.  Extensive  experiments  have  been  conducted  on  human-object  interaction  datasets.  The 
experimental  results  demonstrate  that  the  proposed  framework  achieves  consistent  improvements 
over  the  state-of-the-art  action  recognition  accuracies  on  the  benchmarks,  without  using  any 
object  annotation.  In  addition,  the  research  team  proposes  to  use  descriptors  at  Higher  Semantic 
levels  in  combination  with  the  low-level  dynamic  ones  for  action  recognition. 

26)  Feature  Quantization 

The  research  team  propose  to  construct  a  supporting  visual  word  table  for  all  visual  words  by 
visual  word  expansion.  Given  the  initial  quantization  result,  multiple  approximate  nearest  visual 
words  are  identified  by  checking  supporting  visual  word  table,  which  benefits  the  retrieval  recall. 
Moreover,  a  matching  verification  scheme  based  on  binary  SIFT  (BSIFT)  is  presented.  The  Li- 
distance  between  original  SIFT  descriptors  is  demonstrated  to  be  well  kept  with  the  metric  of 
Hamming  distance  between  the  corresponding  binary  SIFT  vectors.  With  the  BSIFT  verification, 
false  positive  matches  can  be  effectively  and  efficiently  identified  and  removed,  which  greatly 
improves  the  accuracy  of  large-scale  image  search.  The  proposed  approach  is  evaluated  by 
conducting  partial-duplicate  image  search  on  a  one-million  image  database.  The  experimental 
results  demonstrate  the  effectiveness  and  efficiency  of  the  proposed  scheme. 


27)  Image  Search  Results  Quality  Assessment 

The  research  team  quantitatively  study  and  formulate  the  image  search  result  preference  learning 
problem.  A  novel  framework  and  a  set  of  valuable  features  to  automatically  compare  the  quality 
of  image  search  result  lists  are  proposed.  A  general  preference  learning  model  and  a  query 
dependent  preference  learning  model  are  proposed.  The  proposed  approach  has  been  tested  on  a 
variety  of  applications  including  optimal  search  engine  selection,  merging  of  search  result  lists, 
selecting  the  best  visual  feature  and  reranking  approach  for  each  individual  query,  and 
synonymous  query  suggestion.  Extensive  experimental  results  have  demonstrated  the 
effectiveness  of  the  proposed  approach  and  its  promising  applications  on  reranking  feature  and 
model  selection,  merging  of  image  search  results,  as  well  as  query  suggestion.  This  work  will 
explicitly  guide  the  research  in  visual  reranking  ability  estimation  and  provide  a  path  for  query 
difficulty  modeling. 

28)  Image  Search  Reranking 

The  research  team  introduce  a  learning-based  reranking  method  “Topic  Aware  Reranking” 
(TARerank)  to  refine  text-based  image  search  results.  This  method  not  only  takes  topic 
importance  into  consideration,  but  also  directly  learns  a  reranking  model  by  optimizing  a 
criterion  related  to  reranking  performance  in  terms  of  both  relevance  and  diversity  in  one  stage 
simultaneously.  To  better  model  the  hierarchical  topic  structure  of  search  results  and  describe  the 
relevance  and  diversity  in  one  criterion  seamlessly,  NCTC  is  also  proposed  to  quantify  the 
hierarchical  TC.  Compared  with  the  two-step  optimization  in  other  diversified  reranking  methods, 
TARerank  can  achieve  the  joint  optimum  of  improving  relevance  and  diversity.  Besides,  the 
learning  procedure  can  bridge  the  gap  between  low-level  visual  feature  diversity  and  high-level 
semantic  topic  diversity  to  some  extent.  These  two  advantages  ensure  the  superiority  of 
TARerank.  By  conducting  extensive  experiments  on  a  Web  image  dataset,  the  research  team  has 
demonstrated  the  effectiveness  of  the  proposed  method.  This  method  will  be  a  promising  new 
paradigm  for  Web  image  search  reranking. 

29)  Sketch-based  3D  Model  Retrieval 

In  this  work,  the  research  team  propose  and  implement  a  novel  3D  sketching  virtual  drawing 
“board”  (software),  which  allows  users  to  freely  draw  3D  sketches  in  the  air  (a  real  3D  space). 
Based  on  this  developed  3D  sketching  virtual  drawing  board,  the  first  human  3D  sketch  dataset  is 
collected.  The  research  team  also  introduce  a  3D  sketch-based  3D  model  retrieval  system  to 
solve  the  matching  problem  between  3D  sketches  and  models.  And  the  proposed  approach  shows 
promising  application  potential  for  3D  sketch  understanding  or  recognition,  large  scale  3D  model 
search,  and  on-line  3D  model  shopping,  etc. 

This  work  is  the  first  attempt  to  explore  3D  sketching  in  a  3D  space  and  to  develop  an  innovative 
retrieval  system  that  enables  users  to  search  3D  models  based  on  hand-drawn  3D  sketches.  The 
implications  of  this  work  could  be  tremendous,  as  3D  sketching  allows  for  more  direct 
communication  in  a  user’s  drawing,  which  could  not  only  enhance  3D  model  retrieval  accuracy, 


but  also  provide  a  possibility  for  other  human  sketch  related  applications,  such  as  virtual  try-on 
systems  for  clothes,  glasses  and  watches.  This  work  will  also  explicitly  guide  the  research  in  3D 
sketching  and  provide  a  path  for  large  scale  sketch-based  image,  video,  and  object  retrieval. 


30)  3D  Symmetry  Detection 

The  research  team  propose  a  novel  and  efficient  view-based  symmetry  detection  algorithm.  The 
proposed  algorithm  can  find  symmetry  plane(s)  by  matching  the  viewpoint  entropy  features  of  a 
set  of  sample  views  of  a  3D  model  aligned  beforehand  using  Continuous  Principal  Component 
Analysis  (CPC A).  The  experimental  results  show  the  proposed  symmetry  detection  algorithm  is 
more  accurate  (in  terms  of  both  the  positions  of  detected  symmetry  planes  and  sensitivity  to 
minor  symmetry  differences),  efficient,  robust  (e.g.  to  the  number  of  vertices  and  parameter 
settings  such  as  view  sampling),  and  versatile  in  finding  symmetry  planes  of  diverse  models. 
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Simulated  Range  Images  Competition”,  and  Second  Place  in  “Retrieval  of  Objects 
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